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OVERVIEW 


The  Industry/Government  Open  Systems  Specification  (IGOSS)  Partners  (i.e.,  the  U.S. 
Federal  Government,  the  Canadian  government,  Manufacturing  Automation  Protocol 
(MAP)  User  Group,  the  Technical  and  Office  Protocol  (TOP)  User  Group,  and  the 
electric  power  industry)  have  developed  a number  of  networking  profiles  based  on 
Open  Systems  Interconnection  (OSI)  standards.  The  resulting  National  Institute  of 
Standards  and  Technology  (NIST)  Special  Publication  500-217  which  specifies  the 

Industry/Government  Open  Systems  Specification  (IGOSS)  [NIST  15],  leads  to  the 
establishment  of  procedures  to  assess  which  networking  products  conform  to  the 
IGOSS  profiles  and  which  are  interoperable. 

This  document  describes  the  procedures  for: 

• the  registration  and  review  of  IGOSS  abstract  test  suites; 

• the  assessment  of  IGOSS  test  tools; 

• the  accreditation  of  IGOSS  testing  laboratories; 

• the  registration  of  IGOSS  tested  products;  and 

• the  assessment  and  registration  of  organization  providing  IGOSS  interoperability 
testing. 

The  eventual  usage  of  such  registers  is  beyond  the  scope  of  this  document  and  is 
expected  to  appear  in  the  Acquisition  Statement  of  each  IGOSS  partner. 

To  achieve  this  goal,  this  document,  which  supersedes  NISTIR  4594,  references  other 
publicly  accessible  registers,  other  publications,  and  works  conducted  under  the 
auspices  of: 

• the  National  Voluntary  Laboratory  Accreditation  Program, 

• the  Open  Systems  Environments  (OSE)  Implementors'  Workshop, 

• the  IGOSS  Panel  or  its  designated  Agent,  and 

• the  International  Organization  for  Standardization  (ISO). 

The  IGOSS  Testing  Framework  addresses  the  following  Issues: 

• IGOSS  Conformance  Testing:  IGOSS  Conformance  Testing  shall  be 
conducted  by  accredited  test  laboratories  using  assessed  and  registered  Means 
of  Testing.  Registered  results  of  conformance  testing  will  be  published 
periodically  for  purposes  of  Industry/Government  procurement.  Mutual 
recognition  of  other  conformance  testing  authorities'  results  will  be  considered 
using  these  criteria  as  a basis. 
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• IGOSS  Interoperability  Testing:  For  suppliers  claiming  IGOSS  conformance, 
interoperability  testing  against  IGOSS  compliant  peer-entities  is  advised  for 
IGOSS  application  and  relay  stacks.  Criteria  and  procedures  are  identified  to 
select  and  register  Interoperability  Testing  and  Registration  Services. 

• Laboratory  Accreditation:  Accreditation  Policies  and  procedures  are  published 
under  the  auspices  of  the  National  Voluntary  Laboratory  Accreditation  Program 
(NVLAP)  and  include:  policy,  procedures  and  criteria  to  determine  that  candidate 
test  laboratories  are  qualified  to  conduct  IGOSS  product  testing;  procedures  and 
criteria  to  determine  that  registered  test  methods  are  employed  by  candidate  test 
laboratories.  Mutual  recognition  of  other  Accreditation  authorities'  results  will  be 
considered  using  these  criteria  as  a basis.  This  report  does  not  distinguish 
between  a conformance  testing  laboratory  which  Is  first  party  (self-testing)  or 
third  party  (independent  of  product  supplier). 

• Abstract  Test  Suites:  Criteria  identified  for  test  suite  coverage  in  this  report 
shall  be  applied  to  identify  or  develop,  amend  as  necessary,  and  maintain  a set 
of  Abstract  Test  Suites  for  IGOSS.  Abstract  test  suites  registered  by  the  NIST 
shall  be  used  as  the  standard  reference  for  the  assessment  of  Means  of  Testing. 

• Means  of  Testing:  Assessment  of  means  of  Testing  (MOT)  shall  be  conducted 
by  accredited  test  laboratories  approved  by  the  Agent  of  the  IGOSS  Panel. 
Registered  MOTs  will  be  published  periodically.  Mutual  recognition  of  other 
MOT  assessment  authorities'  results  will  be  considered  using  these  criteria  as  a 
basis. 

• Public  Registers:  Registers  will  be  maintained  and  published  periodically  for  the 
following: 

1)  PICS  Proform ae; 

2)  Abstract  Test  Suites; 

3)  Reference  Implementations  for  MOT  Assessment; 

4)  Qualified  Means  of  Testing; 

5)  Accredited  Laboratories; 

6)  Successfully  Conformance  Tested  IGOSS  Products; 

7)  Interoperability  Test  Suites;  and 

8)  Interoperability  Testing  and  Registration  Services. 
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General  Requirements  for  Testing  IGOSS 


1.  INTRODUCTION 

Acceptance  of  vendor  products  for  use  within  Industry/Government  operations  is  the 
responsibility  of  an  Industry/Government  Acquisition  Authority.  Normally,  an  Acquisition 
Authority  develops  an  acceptance  test  plan  to  evaluate  the  functional  and  performance 
characteristics  of  proposed  products  against  requirements  specified  within  a request  for 
proposal  (RFP).  When  an  Acquisition  Authority  introduces  a requirement  for  standards 
compliance  within  an  RFP,  a new  testing  issue  is  created  - determining  compliance  of 
proposed  products  with  the  standard.  When  a data  communication  profile,  such  as  the 
Industry/Government  Open  Systems  Specification  (IGOSS)  [NIST  25]  is  cited,  an 
additional  testing  issue  arises  - determining  interoperability  between  proposed  products 
and  existing  products  that  are  known  to  comply  with  the  IGOSS. 

The  IGOSS  Testing  Framework  is  Intended  to  provide  the  Acquisition  Authority  with  as 
much  assistance  as  possible  to  determine  compliance  to  IGOSS  and  to  demonstrate 
interoperability  between  vendor  products  purporting  to  comply  with  IGOSS.  The 
Acquisition  Authority  should  reserve  the  right  to  test  proposed  products  against  more 
stringent  criteria  should  the  need  exist  and  should  the  cost  be  justified.  In  such  cases, 
the  following  IGOSS  test  policy  will  provide  a significant  foundation. 


1.1  Background 

The  Industry/Government  Open  Systems  Specification  (IGOSS)  is  jointly  authored  by 
the  U.S.  Government,  the  Canadian  government.  Manufacturing  Automation  Protocol 
(MAP)  User  Group,  the  Technical  and  Office  Protocol  (TOP)  User  Group,  and  the 
electric  power  industry.  Each  of  these  five  major  user  organizations  have  previously 
Issued  their  own  procurement  profiles  to  coordinate  the  acquisition  and  operation  of 
computer  networking  products  and  services  based  on  the  international  Open  Systems 
Interconnection  (OSI)  standards. 

The  MAP  specification  [MISC  1]  was  first  published  by  the  General  Motors  Corporation 
in  1984.  Other  companies  joined  the  effort  to  promote  the  use  of  the  OSI  protocols  by 
the  factory  automation  community.  Under  the  leadership  of  General  Motors,  the  MAP 
Users  Group  was  formed  which  supported  General  Motors  in  the  work  of  producing 
subsequent  versions  of  the  MAP  specification.  Around  the  same  time,  the  office  and 
engineering  community  recognized  the  importance  of  developing  an  OSI  procurement 
specification  which  would  accelerate  the  availablility  of  off-the-shelf  computer 
networking  products  that  would  meet  the  needs  of  users.  The  first  version  of  the  TOP 
specification  [MISC  2]  was  published  in  1985  by  the  Boeing  Corporation  and,  under 
Boeing's  leadership,  the  TOP  Users  Group  was  formed  in  the  latter  part  of  that  year. 
The  MAP  and  TOP  communities  joined  forces  to  coordinate  their  activities  under 
common  North  American  and  World  Federation  of  MAP/TOP  Users  Groups.  The  MAP 
and  TOP  specifications  are  now  maintained  by  the  World  Federation  of  MAP/TOP 
Users  Groups.  The  Corporation  for  Open  Systems  (COS)  distributes  both  the  MAP  and 
TOP  documents.  The  MAP  and  TOP  organizations  also  jointly  organized  the 
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Enterprise  Networking  Event  in  1988  at  which  52  vendors  demonstrated  that  OSI 
products  can  be  used  to  solve  real  business  problems. 

In  late  1986  the  National  Bureau  of  Standards  (NBS),  now  the  National  Institute  of 
Standards  and  Technology  (NIST)  Initiated  development  of  the  Government  Open 
Systems  Interconnection  Profile  (GOSIP).  A Federal  inter-agency  group  of  experts  was 
formed,  and  evolved  Into  the  GOSIP  Advanced  Requirements  Group,  that  now  has 
responsibility  for  promulgating  each  new  version  of  the  GOSIP.  The  NIST  chairs  this 
group  and  has  editing  responsibility  for  the  document.  At  the  time  the  first  draft  of 
GOSIP  was  written,  the  MAP  and  TOP  specifications  were  nearing  stability,  vendor  OSI 
implementations  were  being  successfully  demonstrated,  and  commercial  OSI  products 
were  just  entering  the  marketplace.  The  intent  of  GOSIP  is  to  transmit  Federal  user 
requirements  to  vendors  and  to  encourage  vendors  to  build  OSI  products  satisfying 
those  requirements.  The  GOSIP,  unlike  the  MAP  and  TOP  documents.  Is  a mandate. 
GOSIP  mandates  that  Federal  agencies  acquire  OSI  products  when  acquiring  the 
services  provided  by  the  OSI  protocols  referenced  in  the  document.  Since  the  GOSIP 
must  be  referenced  in  Federal  procurement  requests,  where  applicable,  GOSIP 
contains  only  those  OSI  protocols  which  are  expected  to  be  implemented  in  vendor 
products.  The  MAP  and  TOP  documents  included  some  specifications  of  OSI  protocols 
which  those  communities  wanted  the  vendors  to  implement  in  the  future.  Accordingly, 
although  informal  coordination  has  existed  between  the  MAP/TOP  and  Federal 
communities,  the  protocols  in  GOSIP  have  tended  to  be  a subset  of  the  protocols  in  the 
union  of  the  MAP  and  TOP  documents. 

In  order  to  promote  Interoperability  among  computer  systems  supplied  to  the  electric 
power  industry,  the  Electric  Power  Research  Institute  (EPRI)  initiated  the  Utility 
Communications  Architecture  (UCA)  project.  The  first  phase  of  the  project  identified 
the  Information  requirements  within  an  electric  utility.  Subsequent  phases  identified 
appropriate  standards  for  inclusion  in  Version  1 of  the  UCA  specification.  The  UCA 
document,  like  the  MAP  and  TOP  documents,  is  a specification  of  user  requirements. 
Twenty  utility  companies  participated  in  the  review  of  the  draft  document,  which  was 
then  formally  released  to  the  vendor  community.  EPRI  continues  to  have  the 
responsibility  for  maintaining  the  UCA  specification.  OSI  protocols  are  the  foundation 
on  which  Version  1 of  the  UCA  specification  is  based.  The  UCA  authors  were 
knowledgeable  of  the  MAP,  TOP,  and  GOSIP  documents  and  recognized  the 
Importance  of  aligning  specifications  so  that  vendors  would  not  be  forced  to  build  a 
different  set  of  products  for  each  new  user  community. 

In  April  1987,  the  Canadian  federal  government,  announced  a new  policy  on  OSI.  This 
policy,  which  applies  to  all  Canadian  government  departments,  endorses  OSI  as  a 
information  Technology  (IT)  strategy  in  preference  to  any  manufacturer-specific  or 
installation-specific  architecture  and  requires  that  departments  and  agencies  state  a 
clear  preference  for  OSI-based  products  and  services  in  their  procurements.  In  order 
to  assist  users  to  migrate  to  OSI,  work  began  in  1987  to  develop  the  Canadian  Open 
Systems  Application  Criteria  (COSAC).  This  work  is  led  and  coordinated  by  the 
Treasury  Board  Secretariat  (TBS)  which  is  responsible  for  Canadian  government  IT 


Version  1.0 


June  1,  1994 


Page  2 


General  Requirements  for  Testing  IGOSS 


standards  and  policy.  COSAC  comprises  endorsements  of  the  OSI  based  standards, 
OSI  functional  profiles,  and  guidance  documents,  all  of  which  are  published  as 
Treasury  Board  Information  Technology  Standards.  In  producing  COSAC,  maximum 
alignment  with  other  government  and  international  specifications  has  also  been  an 
objective.  Thus,  cooperation  between  the  five  user  communities  to  produce  the  IGOSS 
is  just  a formal  extension  to  what  has  existed  informally  for  some  time. 

This  specification  is  the  standard  reference  for  all  IGOSS  organizations  to  use  when 
acquiring  and  operating  ADP  systems  or  services  and  communications  systems  or 
services  intended  to  conform  to  Open  Systems  Interconnection  protocols.  This 
specification  will  allow  major  network  users  In  Canada  and  the  United  States  to 
consolidate  their  procurement  and  operational  requirements  in  a single  document  and 
is  expected  to  be  welcomed  by  OSI  vendors  because  it  implicitly  represents  significant 
purchasing  power. 

1.1.1  Evolution  of  the  IGOSS 

An  IGOSS  Panel,  which  consists  of  members  from  each  IGOSS  organization,  is 
responsible  for  creating  a procurement  specification  that  meets  the  common 
requirements  of  the  participating  user  groups.  Since  the  IGOSS  will  be  referenced  in 
procurement  requests,  the  document  can  only  Include  functionality  which  vendors  are 
In  the  process  of  implementing  or  have  already  implemented. 

For  this  version  and  all  subsequent  versions  of  the  IGOSS,  the  IGOSS  Panel,  after 
consulting  with  members  of  their  respective  organizations,  recommends  the  protocols 
and  services  to  be  included  In  the  common  procurement  specification.  In  making  this 
decision,  the  IGOSS  Panel  considers  the  progress  made  in  developing  the  standards 
and  implementors  agreements  and  the  commitment  of  the  vendors  to  develop  products 
based  on  these  documents.  The  IGOSS  Panel  members  are  then  responsible  for 
obtaining  formal  concurrence  on  the  proposed  content  of  the  draft  document  from  the 
organizations  that  they  represent.  Members  of  IGOSS  organizations,  as  well  as  industry 
and  government  reviewers,  will  use  a ninety-day  public  comment  period  as  the 
mechanism  to  comment  on  the  document.  The  IGOSS  Panel  will  then  modify  the  draft 
version  of  the  IGOSS,  incorporating  those  comments  which  are  consistent  with  the 
objectives  of  the  IGOSS  organizations.  The  panel  members  will  then  obtain  final 
approval  of  the  document  from  their  respective  organizations  before  publishing  the 
document  in  final  form.  (The  IGOSS  will  be  published  as  a NIST  special  publication  In 
the  United  States  and  as  a Treasury  Board  IT  Standard  in  Canada.)  This  approval 
process  will  apply  when  each  new  version  of  the  IGOSS  is  issued. 

IGOSS  will  be  updated  by  issuing  new  versions  at  appropriate  Intervals,  tentatively 
every  two  years,  to  reflect  the  progress  being  made  by  vendors  in  providing  OSI 
products  with  new  services  for  government  and  commercial  uses.  A new  version  of 
IGOSS  will  supersede  the  previous  version  of  the  document.  Every  attempt  will  be 
made  to  obtain  backward  compatibility  with  the  previous  version  of  the  document. 
Every  new  version  of  IGOSS  will  specify  the  architecture  and  protocols  that  were 
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included  in  each  of  the  previous  versions  so  that  the  protocols  added  to  each  version 
can  easily  be  determined. 

1.1.2  Scope  of  IGOSS 

In  an  Increasingly  complex  world,  the  need  to  exchange  information  has  become  an 
ever  more  Important  factor  In  conducting  business.  Until  recently,  computer  networking 
technology  has  not  kept  pace  with  this  need  to  communicate.  Even  now,  many  users 
have  "islands"  of  computer  systems  built  by  different  vendors,  or  even  by  the  same 
vendor,  that  cannot  exchange  information.  The  IGOSS  indicates  that.  In  response  to 
this,  the  vendor  community  has  developed  a non  proprietary  solution  for  this 
requirement  to  exchange  Information.  The  solution  uses  OSI  protocols,  allowing 
computer  systems  built  by  different  vendors  to  exchange  data.  These  OSI  protocols 
give  users  access  to  standardized  applications  which  can  operate  over  diverse  reliably 
interconnected  subnetworks.  The  IGOSS  lists  the  protocol  specifications  and  provides 
procurement  alternatives. 

IGOSS  significantly  expands  the  scope  of  user  services  provided  by  OSI  applications. 
The  IGOSS  electronic  mall  service  uses  the  OSI  standard  for  Message  Handling 
Systems  (MHS).  There  are  two  major  MHS  components:  the  Message  Transfer 
System  (MTS)  and  the  cooperating  user  agents.  IGOSS  specifies  two  types  of  user 
agents  for  which  International  Standardized  Profiles  are  currently  under  develpment. 
Additional  user  agent  (UA)  types  may  be  specified  by  procurers.  The  two 
Internationally  standardized  User  Agents  are  the  Interpersonal  Messaging  User  Agent, 
used  to  send  a personal  message  from  an  originator  to  one  or  more  recipients,  and  the 
Electronic  Data  Interchange  User  Agent,  used  to  send  and  receive  business  related 
tranactions  using  standard  transaction  sets.  File  transfer  services  are  provided  by  the 
File  Transfer,  Access,  and  Management  (FTAM)  application.  A remote  terminal  access 
capability  is  provided  by  the  Virtual  Terminal  (VT)  application.  The  Directory  Services 
application  provides  access  to  a distributed  directory  on  behalf  of  human  users  or  OSI 
applications  such  as  MHS  or  FTAM.  The  Remote  Database  Access  (RDA)  application 
allows  the  interconnection  of  database  applications  resident  In  heterogeneous 
environments.  The  Transaction  Processing  (TP)  application  provides  for  reliable 
support  of  distributed,  interdependent  transactions.  The  X-Windows  application  allows 
a user  to  gain  access  to  multiple  computer  applications  by  dividing  a screen  into 
multiple  sections,  each  section  responding  independently  to  Input  from  a keyboard  or  a 
pointing  device  or  both.  The  Manufacturing  Messaging  Specification  (MMS)  application 
allows  objects  related  to  a process  control  environment  to  be  accessed  and 
manipulated  across  a network.  The  Information  Retrieval  (IR)  application  provides 
Information  search  and  retrieval  by  converting  queries  constructed  in  a local  query 
language  into  a common  representation.  All  of  these  applications  use  lower  layer  OSI 
protocols  to  guarantee  that  end  systems  attached  to  subnetwork  technologies  [e.g., 
X.25  Wide  Area  Network  (WAN),  Local  Area  Network  (LAN),  Integrated  Services  Digital 
Network  (ISDN),  Frame  Relay]  can  interoperate. 
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1.1.3  Relationship  of  the  IGOSS  to  Existing  Profile  Documents 

The  IGOSS  is  a collaborative  effort  of  organizations  that  have  previously  published  the 
Canadian  Open  System  Application  Criteria,  the  Manufacturing  Automation  Protocol 
specification,  the  Technical  and  Office  Protocol  specification,  the  United  States 
Government  Open  Systems  Interconnection  Profile  and  the  Utility  Communications 
Architecture  documents.  These  documents  will  continue  to  be  published  by  the 
responsible  IGOSS  organizations,  but  now  they  will  primarily  refer  to  the  IGOSS  to 
specify  the  OSI  procurement  requirements  for  each  organization.  The  documents  will 
also  contain  specifications  for  any  protocol  required  by  the  organization,  but  not  agreed 
to  in  common,  and  an  applicability  statement  which  Indicates  how  the  IGOSS  must  or 
should  be  used.  The  Federal  Government  will  continue  to  promulgate  the  Government 
Open  Systems  Interconnection  Profile  as  a Federal  Information  Processing  Standard 
(FIPS).  In  Canada,  the  Canadian  Open  Systems  Application  Criteria  will  continue  to  be 
developed  as  a Treasury  Board  IT  Standard  (TBITS). 

1.1.4  Applicability 

The  IGOSS  specifies  a set  of  OSI  protocols  for  computer  networking  that  is  intended  for 
acquisition  and  use  by  IGOSS  organizations.  Each  IGOSS  organization  will  specify  the 
applicability  of  IGOSS  to  Its  own  members.  The  detailed  statements  of  applicability  will 
appear  in  supplementary  profile  documents,  issued  by  each  IGOSS  organization. 

1.1.5  IGOSS  Functionality 

Version  2 of  GOSIP  was  the  base  document  used  to  prepare  the  IGOSS.  All  GOSIP 
Version  2 protocols  are  included  In  the  IGOSS.  The  functionality  added  to  the  base 
document  to  form  Version  1 of  the  IGOSS  is  as  follows: 

1.  Message  Handling  Systems  (CCITT  1988  Recommendation); 

2.  Electronic  Data  Interchange  (EDI)  User  Agent; 

3.  File  Transfer,  Access,  and  Management  (FTAM)  - (Phase  3); 

4.  Virtual  Terminal  Service  (S-mode  Paged  and  X.3  profiles); 

5.  Directory  Services; 

6.  Remote  Database  Access; 

7.  Transaction  Processing; 

8.  Manufacturing  Message  Specification; 

9.  X-Windows  over  OSI; 

10.  Information  Retrieval; 

11.  Fiber  Distributed  Data  Interface  (FDDI); 

12.  Frame  Relay; 

13.  Point  to  Point  Protocol  (PPP); 

14.  Intermediate  System-Intermediate  System  routing  (IS-IS)  protocol; 

15.  Inter-Domain  Routing  Protocol  (IDRP); 

16.  Network  Management  protocols;  and 

17.  Connectionless  Upper  Layer  services. 

18.  Minimal  OSI  Upper  Layer  Services 
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1.1.6  Sources  of  Protocols  Specification 
1.1. 6.1  Primary  Source 

Relationship  of  the  IGOSS  Specifications  to  Workshop  Agreements 

The  primary  source  of  protocol  specifications  in  the  IGOSS  Is  the  Stable 
Implementation  Agreements  for  Open  Systems  Interconnection  Protocols  [NIST  1], 

hereafter  referred  to  as  the  Workshop  Agreements.  By  primary  source,  it  is  meant  that 
where  the  IGOSS  uses  a given  protocol,  it  cites  that  protocol  by  reference  to  the 
Workshop  Agreements.  The  primary  source  is  used  in  ail  instances  where  the  protocol 
of  interest  has  been  specified  In  the  Workshop  Agreements.  Section  4 of  this 
specification  augments  those  agreements  when  necessary  to  provide  the  functionality 
required  by  IGOSS  organizations. 

The  primary  source  document  was  created  and  Is  maintained  by  the  Open  Systems 
Environment  (OSE)  Implementors  Workshop  (01 W).  The  Workshop  Agreements 

provide  implementation  specifications  that  are  derived  from  service  and  protocol 
standards  issued  by  the  International  Organization  for  Standardization  (ISO)  and  the 
Consultative  Committee  for  International  Telegraphy  and  Telephony  (CCITT).  A copy 
of  the  Workshop  Agreements  Is  essential  to  thoroughly  understand  the  material  in  this 
document. 

A new  version  of  the  Workshop  Agreements  Is  created  each  year,  following  the 
December  OSE  Implementor's  Workshop  meeting,  if  a sufficient  amount  of  new 
functionality  has  accumulated  since  the  previous  version  was  Issued.  It  is  the  intent  of 
the  Workshop  that  new  versions  of  the  Workshop  Agreements  be  backwardly 
compatible  with  previous  versions.  Replacement  pages  applying  to  the  latest  version  of 
the  Workshop  Agreements  may  be  published  at  regular  intervals  during  the  year. 
These  replacement  pages  may  contain  errata  to  the  original  stable  agreements  that  are 
approved  by  the  Workshop  plenary.  The  latest  replacement  pages  are  distributed  to  all 
workshop  attendees  and  are  available  through  several  sources.  (See  NIST  Reference  1 
for  ordering  Information.) 

Each  new  version  of  the  IGOSS  will  reference  the  latest  appropriate  version  of  the 
Workshop  Agreements  as  the  base  document.  These  agreements,  although  stable, 
can  be  modified  by  errata  which  correct  technical  and  editorial  mistakes  or  by  changes 
which  are  required  to  align  with  evolving  international  standards  or  agreements 
developed  in  other  regional  workshops.  These  changes  to  the  Workshop  Agreements 
are  stabilized  each  December  and  become  effective  as  an  implementation  requirement 
the  following  December  (e.g.,  the  stabilized  Workshop  Agreement  for  December,  1993 
becomes  effective  as  an  implementation  requirement  as  of  December,  1994).  This 
becomes  effective  with  the  December,  1993  Workshop  Agreements. 

Each  version  of  the  IGOSS  is  issued  in  draft  form  for  public  comment  before  it  Is  issued 


Version  1.0 


June  1, 1994 


Page  6 


General  Requirements  for  Testing  IGOSS 


in  final  form.  Final  editions  of  the  IGOSS  will  reference  only  the  Stable  Workshop 
Agreements. 

Relationship  of  the  IGOSS  Specifications  to  International  Standardized  Profiles 

International  Standardized  Profiles  (ISPs)  are  functional  profiles  which  are  approved  for 
publication  by  the  ISO/IEC  JTC1  Special  Group  on  Functional  Standards  (SGFS)  under 
SGFS  procedures.  These  functional  profiles  should  be  technically  harmonized  at  the 
regional  workshop  level  before  submission  to  the  SGFS. 

It  is  a goal  that  the  IGOSS  technical  specifications  reference  ISPs,  when  possible.  The 
linkage  by  which  this  will  occur  will  be  the  Workshop  Agreements.  For  each  protocol, 
the  appropriate  Special  Interest  Groups  will  determine  if  and  when  it  Is  appropriate  to 
replace  the  existing  Workshop  Agreement  text  with  a reference  to  an  ISP. 

1.1. 6.2  Secondary  Sources 

The  IGOSS  must  be  complete  in  that  open  systems  procured  in  accordance  with  it  must 
interoperate  and  must  provide  service  generally  useful  for  government  and  commercial 
computer  networking  applications.  The  Workshop  Agreements  continue  to  evolve,  but 
remain  incomplete.  (The  appendices  of  the  IGOSS  cite  needed  work.)  Thus,  where 
the  Workshop  Agreements  are  not  complete,  the  IGOSS  may  augment  protocol  and 
service  specifications  from  the  following  sources. 

0 International  Standards  and  Recommendations 
0 Draft  International  Standards 

0 Institute  of  Electrical  and  Electronics  Engineers  (IEEE)  Standards 
0 Working  Implementation  Agreements  from  the  OIW 

Since  this  profile  is  one  of  open  systems,  the  secondary  sources  include  specifications 
that  are  international  standards  or  are  advancing  to  become  International  standards. 
They  are  included  in  the  IGOSS,  where  needed,  to  help  satisfy  the  criterion  of  utility. 
Note  that  secondary  sources  exclude  protocols,  however  mature,  that  are  not  a part  of 
the  International  standards  process. 

1. 1.6.3  Tertiary  Sources 

Even  the  secondary  sources  named  above  may  not  provide  a complete  and  useful 
networking  system  today.  It  may  be  necessary  for  the  IGOSS  to  augment  protocol  and 
service  specifications  from  the  following  sources. 

0 National  Standards 
0 Government  Standards 
0 Military  Standards 
0 Other  publicly  available  specifications 

The  use  of  specifications  from  other  than  the  primary  and  secondary  sources  is 
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undesirable.  It  is  expressly  intended  that  these  omissions  from  standards  work  be 
brought  to  the  attention  of  the  international  standards  bodies  so  that  acceptable 
international  standards  may  be  developed  as  rapidly  as  possible.  The  IGOSS  Panel 
will  replace  all  tertiary  source  protocols  in  the  IGOSS  with  suitable  primary  and 
secondary  sources,  as  soon  as  they  are  available. 

1.1.7  IGOSS  Errata 

All  errata  to  the  IGOSS  will  be  subject  to  the  same  public  review  process  that  exists  for 
the  IGOSS  document.  The  errata  may  take  effect  at  any  time  after  the  public  review 
period  if  approved  by  the  IGOSS  Panel.  Since  each  new  version  of  the  IGOSS  will 
supersede  the  previous  version,  errata  to  previous  versions  will  be  published  in 
subsequent  versions. 

1.2  Purpose  of  IGOSS  Testing  Framework 

The  purpose  of  this  document  is  to  provide  the  framework  for  uniform 
Industry/Government-wide  procurement  of  IGOSS  conformant  products.  As  such,  this 
document  is  Intended  to  inform  government  agencies,  industry,  standards  development 
bodies,  and  other  interested  organizations  of  the  IGOSS  policy  with  regard  to 
conformance  testing  and  interoperability  testing  of  IGOSS  products. 

The  objectives  of  the  IGOSS  Testing  Framework  are: 

To  reduce  the  overall  information  systems  costs  by  making  It  easier  and  less 
expensive  to  maintain  information  technology  applications  and  to  transfer  these 
applications  among  different  Information  systems.  Including  replacement 
systems; 

To  protect  the  technical  assets  and  staff  time  of  the  Industry/Government  by 
insuring  to  the  extent  possible  that  purchased  products  comply  with  IGOSS; 

To  Identify  test  methods  and  competent  test  laboratories  for  assisting 
Industry/Government  In  the  procurement  of  industry  supplied  IGOSS  products; 

To  Increase  the  likelihood  of  interoperability  of  IGOSS  conformant  products. 

Further  detailed  solutions  to  the  guidelines  in  this  document  are  given  in  a companion 
handbook: 

The  NVLAP  Program  Handbook:  Operational  Requirements  of  the  Laboratory 

Accreditation  Program  for  IGOSS  Conformance  Testing  [NVLAP  1]  and  NIST 
Handbook  150.  National  Voluntary  Laboratory  Accreditation  Program.  Procedures  and 

General  Requirements  [NVLAP  2]  provides  the  administrative  procedures  for  NVLAP 
accreditation. 
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The  see  Handbook,  Information  Technology  and  Telecommunications  (IT&T) 
Laboratory  Accreditation  Program  [See  5]  provides  the  administrative  procedures  for 
see  Accreditation. 


1.3  Scope 

This  document  provides  detailed  advisory  provisions  with  respect  to  conformance  and 
interoperability  testing  given  in  ''Industry/Government  Open  Systems  Specification 
(IGOSS)  Version  1.0".  This  document  defines  policy  and  procedures  related  to 
conformance  testing  and  interoperability  testing  for  IGOSS.  Other  types  of  testing  such 
as  performance,  acceptance,  and  quality  testing,  are  not  addressed. 

In  determining  testing  requirements  for  IGOSS,  a number  of  areas  are  considered: 
Industry/Government  testing  needs,  test  method  technology,  standard  specifications, 
alternative  testing  sources  (third-party  testing,  Industry/Government  testing,  self-testing, 
etc.),  and  existing  accreditation  and  certification  systems. 

The  policy  and  procedures  for  conformance  testing  and  for  interoperability  testing 
defined  herein  apply  whenever  IGOSS  standards  are  required  to  support 
Industry/Government  objectives  for  information  systems. 

The  report  is  addressed  to: 

1)  Procurement  Authorities  Intending  to  procure  OSI  products; 

2)  Suppliers  of  OSI  products  wishing  to  market  to  the  IGOSS  Partners; 

3)  Suppliers  of  OSI  test  services  seeking  accreditation  as  a test  laboratory; 

4)  Developers  of  the  means  of  testing  OSI  products  wishing  to  supply  to  accredited 
laboratories. 

5)  Suppliers  of  OSI  interoperability  testing  and  registration  services  seeking 
recognition  by  the  IGOSS  Partners. 

The  program  of  registration  will  be  administered  by  the  IGOSS  Panel,  or  Its  appointed 
agent.  In  the  text  that  follows  the  acronym  "Agent  of  the  IGOSS  Panel*"  Is  used  to 
mean  "the  Agent  of  the  IGOSS  Panel  or  any  organization  supporting  the  Agent  of  the 
IGOSS  Panel". 

1.4  Overview  of  Testing 

This  document  is  concerned  with  conformance  and  interoperability  testing  from  the 
point  of  view  of  both  their  conduct  and  the  evaluation  of  their  methods.  To  eliminate 
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confusion  over  which  role  is  being  addressed  this  document  draws  the  distinction 
between  the  terms  assessment  and  accreditation  on  the  one  hand,  and  testing  on  the 
other. 

Testing  means  using  tools,  facilities  and  procedures  to  establish  that  implementations 
of  IGOSS  related  products  are  conformant  and/or  interoperable. 

Means  of  Testing  Assessment  is  the  process  of  determining  that  testing  tools  are  fit 
for  their  declared  purpose  and  of  producing  a MOT  validation/calibration  report. 

Accreditation  is  the  administrative  act  of  recognizing  that:  1)  a test  laboratory  is 
qualified  to  conduct  protocol  testing  after  having  specifically  met  both  the  quality  and 
technical  criteria,  and  2)  the  means  of  testing  employed  by  a test  laboratory  meets 
specific  technical  criteria. 

Registration  is  the  administrative  act  of  recognizing  that  the  tested  products  meet 
specified  criteria  by  registering  the  results  after  successful  testing  has  been  conducted. 
Registration  Is  used  in  lieu  of  issuing  certificates  of  conformity. 

Within  the  International  Organization  for  Standardization  (ISO)  conformance  testing 
methodology  has  developed  and  is  the  subject  of  a separate  standard  (IS  9646  OSI 
Conformance  Testing  Methodology  and  Framework)  [ISO  1 13].  Its  purpose  is  to  define 
standardized  methods  which  may  be  used  for  conformance  testing  and  to  define 
relationships  between:  1)  parties  supplying  the  means  of  testing  OSI  protocols  and  test 
laboratories,  and  2)  test  laboratories  and  their  clients,  and  the  Information  exchanged 
between  them.  Conformance  testing  concentrates  on  determining  whether  an 
implementation  of  a protocol  conforms  to  both  static  and  dynamic  requirements 
specified  in  a protocol  standard. 

Current  conformance  testing  technology  provides  for  tools  which  separate  the  testing 
concerns  of  any  7-layer  OSI  stack  into  three  functional  groups: 

- Upper  Layers:  Session,  Presentation,  Application 

- Intermediate  Layers:  Network,  Transport 

- Lower  Layers:  Physical,  Link 

Higher  layer  protocols  are  tested,  and  operated,  over  a stack  of  supporting  protocols. 
IS  9646  prescribes  testing  of  the  lower  layer  protocols  prior  to  testing  the  protocols 
which  they  support.  One  reason  for  this  arrangement  is  that  direct  access  to  a layer 
service  affords  the  greatest  possible  capability  of  controlling  and  observing  events 
within  that  layer.  Another  reason  is  that  the  development  of  the  means  of  testing 
followed  the  development  of  protocol  stack  implementations,  and  the  means  of  testing 
for  lower-layer  protocols  were  available  earliest.  An  advantage  afforded  by  this 
approach  Is  that  the  Protocol  Conformance  Test  Report  (PCTR)  can  be  used  In  support 
of  incremental  testing,  specifically  so  that  full  regression  testing  is  not  needed  in  testing 
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a larger  stack  which  builds  on  the  already  tested  functionality. 

Full-stack  testing  Is  also  an  alternative,  although  no  7-layer  means  of  testing  is  currently 
in  existence.  Full  stack  methods  require  that  each  protocol  in  the  stack  be  tested  by 
embedded  methods  (except  the  Application  protocols  which  are  tested  by  single-layer 
methods).  This  procedure  is  repeated  for  every  new  Application  stack,  since  each  new 
service  user  may  exercise  paths  within  a lower  layer  protocol  which  are  not  explored  by 
a different  service  user  in  another  full  stack. 

More  recently  Interoperability  testing  has  been  identified  as  a necessary 
complementary  step  in  demonstrating  interworking  of  OSI  implementations.  Whereas 
the  failures  in  conformance  testing  are  likely  to  be  software  errors.  Interworking 
problems  seem  more  likely  to  Include  problems  of  parameter  range  selection,  plus 
attempts  to  use  incompatible  stacks,  attempts  to  use  optional  functions  not 
implemented,  and  failures  to  implement  mandatory  functions.  The  methodology  and 
test  suites  employed  are  subject  to  the  criteria  given  in  clause  6.3.  Since  it  is 
necessary  to  assess  the  interoperability  of  systems,  this  report  identifies  multi-vendor 
bilateral  interoperability  testing. 

1.5  Definitions 

Abstract  Test  Case:  A complete  and  independent  specification  of  the  actions  required 
to  achieve  a specific  test  purpose  (or  a specified  combination  of  test  purposes),  defined 
at  the  level  of  abstraction  of  a particular  abstract  test  method.  It  may  include  a 
preamble  and  postamble  to  ensure  starting  and  ending  In  a stable  state  (i.e.  an 
Identifiable  stable  state  of  the  System  Under  Test  which  can  be  easily  reached  and 
maintained,  such  as  the  'idle'  state  or  the  'data  transfer'  state).  This  specification  may 
Involve  one  or  more  consecutive  or  concurrent  connections. 

Abstract  Test  Method:  The  description  of  how  an  Implementation  Under  Test  is  to  be 
tested,  given  at  an  appropriate  level  of  abstraction  to  make  the  description  independent 
of  any  particular  Implementation  of  testing  tools,  but  with  enough  detail  to  enable  tests 
to  be  specified  for  this  test  method. 

Acceptance  Testing:  Formal  testing  conducted  to  determine  whether  or  not  a system 
satisfies  its  acceptance  criteria  and  to  enable  the  customer  to  determine  whether  to 
accept  the  system.  Formal  testing  may  Include  the  planning  and  execution  of  several 
kinds  of  tests  (e.  g.,  functional,  volume,  performance  tests)  to  demonstrate  that  the 
Implementation  satisfies  the  customer  requirements. 

Accreditation  Body:  An  impartial  body,  governmental  or  non  governmental, 
possessing  the  necessary  competence  and  reliability  to  operate  or  accredit  operation  of 
an  accreditation  system,  and  in  which  the  interests  of  all  parties  concerned  with  the 
function  of  the  system  are  represented. 
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Basic  Interconnection  Tests:  Limited  tests  of  an  Implementation  Under  Test  (lUT)  to 
determine  whether  or  not  there  is  sufficient  conformance  to  the  relevant  protocol(s)  for 
interconnection  to  be  possible,  without  trying  to  perform  thorough  testing. 

Behavior  Tests:  Tests  to  determine  the  extent  to  which  the  dynamic  conformance 
requirements  are  met  by  the  lUT. 

Capability  Tests:  Tests  to  determine  the  capabilities  of  an  lUT.  (Note,  this  involves 
checking  ail  mandatory  capabilities  and  those  optional  ones  that  are  stated  In  the 
Protocol  Implementation  Conformance  Statement  (PICS)  as  supported,  but  not 
checking  those  optional  ones  which  are  stated  in  the  PICS  as  not  supported  by  the 
lUT.) 

Conformance:  In  the  context  of  OSI  a real  system  is  said  to  exhibit  conformance  if  it 
complies  with  the  requirements  of  applicable  OSI  standards  in  its  communication  with 
other  real  systems. 

Conformance  Testing:  Testing  the  extent  to  which  an  lUT  is  a conforming 
implementation. 

Coordinated  Test  Method:  An  external  test  method  for  which  a standardized  test 
management  protocol  is  defined  as  the  test  coordination  procedures,  enabling  the 
control  and  observation  to  be  specified  solely  in  terms  of  the  lower  tester  activity, 
including  the  control  and  observation  of  test  management  PDUs. 

Distributed  Test  Method:  An  external  test  method  in  which  there  is  a PCO  at  the  layer 
boundary  at  the  top  of  the  lUT. 

Dynamic  Conformance  Requirements:  All  those  requirements  and  options  which 
determine  what  observable  behavior  is  permitted  by  the  relevant  OSI  standards  in 
instances  of  communication. 

Dynamic  Interoperability  Requirements:  All  those  requirements  and  options  which 
determine  what  observable  behavior  is  permitted  between  peer  open  systems  by 
compatible  standardized  profiles  of  OSI  standards,  in  instances  of  communication. 

Embedded  Testing:  Testing  the  behavior  of  a single  layer  within  a multi-layer  lUT 
without  accessing  the  layer  boundaries  for  that  layer  within  the  lUT.  (This  is  contrasted 
with  'exposed'  testing  in  which  the  N-service  PCO  of  the  lUT  is  accessible  for  testing.) 

Equivalent  Configuration:  Any  configuration  for  which  conformance  Is  achievable 
using  the  same  registered  test  method  version  used  in  conformance  testing  of  an 
Implementation  under  test. 
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IGOSS  Product:  A product  which  implements  one  or  more  of  the  data  communications 
protocols  identified  in  IGOSS  and  meets  the  requirements  specified  herein. 

Implementation  Under  Test  (lUT):  An  Implementation  of  one  or  more  OSI  protocols  In 
an  adjacent  user/provider  relationship,  being  that  part  of  a real  open  system  which  is  to 
be  studied  by  testing. 

Interconnection:  Establishment  of  communication  between  peer  protocol  entities  over 
a physical  medium  or  an  OSI  layer  service. 

Interoperability:  The  capability  of  one  entity  to  collaborate  with  one  or  more  other  peer 
entities  to  render  a particular  service  to  their  respective  users. 

Interoperability  Test  Case:  An  interoperability  test  case  describes  the  actions  to  be 
carried  out  and  the  rules  for  verdict  assignment  in  order  to  test  for  Interoperability  using 
one  single  well  defined  interoperability  test  purpose. 

Interoperability  Test  Method:  An  interoperability  test  method  describes  the  set  of 
actions  and  associated  procedures  that  guides  a real  open  system  through 
interoperability  evaluation,  with  one  or  more  real  peer  open  systems,  in  order  to 
evaluate  the  degree  of  interoperability  achieved  by  the  systems  involved. 

Interoperability  Test  Purpose:  A prose  description  of  a narrowly  defined  objective  of 
Interoperability  testing,  focusing  on  a single  interoperability  requirement. 

Interoperability  Test  Report:  An  Interoperability  test  report  contains  the  record  of  all 
the  test  results  as  described  in  the  test  records,  together  with  a description  of  the 
environment  in  which  the  tests  have  been  executed. 

Interoperability  Testing:  A process  where  two  or  more  products  execute  the 
interoperability  test  suite  according  to  an  interoperability  test  method. 

Means  of  Testing:  The  realization  of  an  abstract  test  method  as  defined  in  the  OSI 
Conformance  Testing  Methodology  and  Framework.  This  realization  includes  the  test 
system,  executable  test  suite,  testing  support  tools  (hardware  and  software)  and 
documentation  (including  technical  test  procedures). 

Multi-Layer  Testing:  Testing  the  behavior  of  a multi-layer  lUT  as  a whole,  rather  than 
testing  it  layer  by  layer  (In  contrast  to  Single-Layer  Testing). 

National  Voluntary  Laboratory  Accreditation  Program  (NVLAP)  and  Standards 
Council  of  Canada  (SCC):  Voluntary  systems  for  accrediting  laboratories  found 
competent  to  perform  specific  testing  operations.  NVLAP  is  part  of  the  National 
Institute  of  Standards  and  Technology  Office  of  Associate  Director  for  Industry  and 
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Standards.  NVLAP  and  SCC  do  not  confer  product  or  test  data  certification. 

Out-of-Band  Coordination:  A separate  communications  path  used  for  test 
coordination  procedures  which  may  be  realized  from  a lower-layer  service  or  alternative 
physical  media. 

Product  Interoperability  Test  Report:  This  Is  a document  written  at  the  end  of  the 
Interoperability  testing  process,  giving  the  details  of  the  testing  carried  out  for  a specific 
interoperability  test  suite. 

Proficiency  Testing:  Determination  of  laboratory  testing  performance  by  means  of 
comparison  of  tests  on  the  same  or  similar  items  by  two  or  more  laboratories  in 
accordance  with  predetermined  conditions. 

Protocol  Conformance  Test  Report  (PCTR):  A document  written  at  the  end  of  the 
conformance  assessment  process,  giving  the  details  of  the  testing  carried  out  for  a 
particular  protocol.  It  includes  the  Identification  of  the  abstract  test  cases  (if  these  exist) 
for  which  corresponding  executable  test  cases  were  run.  It  also  includes  the  test 
purpose(s)  and  verdict  for  each  test  case. 

Protocol  Implementation  Conformance  Statement  (PICS):  A statement  made  by  the 
supplier  of  an  OSI  implementation,  or  system,  stating  which  capabilities  and  options 
have  been  implemented,  for  a given  OSI  protocol. 

Protocol  Implementation  eXtra  Information  for  Testing  (PIXIT): 

A statement  made  by  a supplier  or  implementor  of  an  lUT  which  contains  or  references 
all  of  the  information  (In  addition  to  that  given  in  the  PICS)  related  to  the  lUT  and  its 
testing  environment,  which  will  enable  the  test  laboratory  to  run  an  appropriate  test 
suite  against  the  lUT. 

Remote  Test  Method:  An  external  test  method  In  which  there  is  neither  a PCO  above 
the  lUT  nor  a standardized  test  management  protocol;  some  requirements  for  test 
coordination  procedures  may  be  implied  or  Informally  expressed  in  the  abstract  test 
suite  but  no  assumption  Is  made  regarding  their  feasibility  or  realization. 

Single-Layer  Testing:  Testing  the  behavior  of  one  layer-protocol  from  a multi-layer 
lUT. 

Static  Conformance  Requirements:  Constraints  which  are  specified  in  OSI  standards 
to  facilitate  interworking  by  defining  the  requirements  for  the  capabilities  of  an 
implementation. 

Static  Interoperability  Requirements:  For  potentially  Interoperable  peers  these 
include: 
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- Compatible  static  conformance  requirements; 

- Both  systems  successfully  conformance  tested; 

- Peers  are  configured  to  enable  Interconnection. 

System  Conformance  Test  Report  (SCTR):  A document  written  at  the  end  of  the 
conformance  assessment  process,  giving  the  overall  summary  of  the  conformance  of 
the  system  to  the  set  of  protocols  for  which  conformance  testing  was  carried  out. 

System  Under  Test  (SUT):  The  real  open  system  in  which  the  lUT  resides. 

Test  System  Environment  Specification:  This  is  a statement  made  by  a supplier  or  an 
implementor  of  an  OSI  product  which  contains  or  references  all  of  the  information  (in 
addition  to  that  given  In  the  PICS)  related  to  the  implementation  and  Its  environment, 
which  will  enable  the  test  parties  to  execute  an  appropriate  test  suite  against  their 
implementations. 

Verdict:  A statement  of  "Pass",  "Fail",  or  "Inconclusive",  specified  in  the  abstract  test 
suite  concerning  conformance  of  an  lUT  with  respect  to  a test  case  that  has  been 
executed. 

1.6  Abbreviations 

ATS'  Abstract  Test  Suite 

CCITT  International  Telephone  and  Telegraph  Consultative  Committee 

CEN  European  Committee  for  Standardization 

CENELEC  European  Committee  for  Electrotechnical  Standardization 

COS  Corporation  for  Open  Systems 

COSAC  Canadian  Open  Systems  Application  Criteria 

CSL  Computer  Systems  Laboratory 

ECITC  European  Committee  for  Information  Technology  Certification 

EPRI  Electric  Power  Research  Institute 

ETCOM  European  Testing  for  Certification  for  Office  and  Manufacturing 

ETS  Executable  Test  Suite 

EWOS  European  Workshop  for  Open  Systems 

FIPS  Federal  Information  Processing  Standard 

GOSIP  Government  Open  Systems  Interconnection  Profile 

lEC  International  Electronical  Commission 

IGOSS  Industry/Government  Open  Systems  Specification 

INTAP  Interoperability  Technology  Association  for  Information  Processing 

IPRL  ISPICS  Requirement  List 
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IS  International  Standard 

ISO  International  Standards  Organization 

ISP  International  Standard  Profile 

ISPICS  International  Standard  Profile  Implementation  Conformance  Statement 
IT  Information  Technology 

IT&T  Information  Technology  and  Telecommunications 

ITU  International  Telecommunication  Union 

ITU-TS  International  Telecommunication  Union  - Telecommunication 

Standardization  Sector  - new  name  for  CCITT 

lUT  Implementation  Under  Test 

JITC  Joint  Interoperability  Test  Center 

MAP  Manufacturing  Automation  Protocol 

NIST  National  Institute  of  Standards  and  Technology 

NVLAP  National  Voluntary  Laboratory  Accreditation  Program 

OIW  OSE  Implementor's  Workshop 

OSE  Open  System  Environment 

OSI  Open  Systems  Interconnection 

OSTC  Open  Systems  Testing  Consortium 

PCO  Point  of  Control  and  Observation 

PCTR  Protocol  Conformance  Test  Report 

PDU  Protocol  Data  Unit 

PICS  Protocol  Implementation  Conformance  Statement 

PIXIT  Protocol  Implementation  eXtra  Information  for  Testing 

see  Standards  Council  of  Canada 

SCTR  System  Conformance  Test  Report 

SIA  Stable  Implementor's  Agreements 

SUT  System  Under  Test 

TOP  Technical  and  Office  Protocol 


2.  ORGANIZATIONAL  MODEL 

Conformance  and  interoperability  testing  for  IGOSS  will  be  accomplished  in  accordance 
with  the  organizational  model  described  In  this  Clause.  This  organizational  model 
consists  of  a Program  Authority,  Program  Sponsor,  Accreditation  Authority,  the  NIST 
OSE  Implementors'  Workshop,  test  laboratories  and  their  clients.  Each  member  of  this 
model  shares  responsibilities  for  assuring  conformance  of  products  to  IGOSS.  Under 
the  rules  and  procedures  established  by  the  IGOSS  Panel,  this  model  will  enable  a 
client  to  have  his  product  tested  by  any  NVLAP  or  SCC  accredited  test  laboratory;  and 
the  test  results  produced  by  that  laboratory  accepted  by  IGOSS  Panel's  Agent  as  the 
basis  for  registration  as  a Conformance  Tested  IGOSS  Product.  The  Product  and 
Means  of  Testing  registration  policy  and  procedures  employed  in  this  organizational 
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model  are  described  in  Annexes  A and  B. 

Full  implementation  or  use  of  all  parts  of  the  organizational  model  depends  on  the 
complexity  of  the  standard(s)  and  the  conformance  testing  methods  applied  for  the 
standard(s).  In  circumstances  deemed  necessary  by  the  Executive  Director  of  the 
IGOSS  Panel,  this  document  and  registers  Identified  by  this  document  may  designate 
alternate  or  supplementary  procedures,  means  of  testing,  and  abstract  test  suites  for 
testing  IGOSS  products. 


Clause  or 
Reference 

Role 

Responsible  Party 

2.1 

Program  Authority 

Executive  Director 
of  the  IGOSS  Panel 

2.1 

Program  Sponsor 

IGOSS  Panel 

2.1 

Program  Operator 

Agent  of  the  IGOSS  Panel 
NIST/CSL 

2.2 

Test  Laboratory  Accreditation 

Authority 

NVLAP  or 

see 

3. 

Laboratory  Accreditation  Procedures 

Agent  of  the  IGOSS  Panel  - 
NIST/CSL,  NVLAP  and  SCC 

4 

Means  of  Testing  Assessment  Authority 

Agent  of  the  IGOSS  Panel 

4 

Annex  B 

Means  of  Testing  Assessment  Procedures 

Agent  of  the 

IGOSS  Panel  - NIST/CSL 

4. 

PICS  Proforma 

OSE  Implementor's 

Workshop 

4.1 

Abstract  Test  Suite  Review  and  Acceptance 

OSE  Implementor's 

Workshop 

2.3 

Provision  of  Means  of  Testing 

Test  Tool  Suppliers 

2.4 

Conformance  Test  Services 

Accredited  Conformance 

Testing  Laboratory 

7.3 

Interoperability  Test  Suite,  Review 
and  Acceptance 

OSE  Implementor's 

Workshop 

7.3 

Multi-Vendor  Interoperability  Testing 

OSI  Product 

Suppliers  & Users 

5. 

PICS  Proforma  Registration 

Agent  of  the  IGOSS  Panel 
NIST/CSL 

5. 

Abstract  Test  Suite  Registration 

Agent  of  the  IGOSS  Panel 
NIST/CSL 

5. 

Interoperability  Test  Suite  Registration 

Agent  of  the  IGOSS  Panel 
NIST/CSL 

5. 

Accredited  Test  Laboratory  Registration 

Agent  of  the  IGOSS  Panel 
NIST/CSL 

4..  5.. 

Annex  B 

Means  of  Testing  Registration 

Agent  of  the  IGOSS  Panel 
NIST/CSL 

5. 

Annex  A 

Conformance  Tested  Product  Registration 

Agent  of  the  IGOSS  Panel 
NIST/CSL 

5. 

Interoperability  Tested  Product  Registration 

Interoperability 

Registration  Authority 

Annex  D 

Procurement  of  IGOSS  Products 

Acquisition 

Authority 

Table  1:  IGOSS  Testing:  Organizational  Responsibilities 
Table  1 provides  a cross-reference  of  the  parties  involved  In  the  model,  together  with 
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their  respective  roles.  The  "Clause  or  Reference"  column  provides  a cross-reference  of 
each  role  with  the  respective  clause  of  this  report  in  which  such  role  is  described. 

2.1  Program  Sponsor/Authority/Agent 

The  IGOSS  Panel  is  the  Program  Sponsor  for  the  IGOSS  conformance  testing 
program.  The  IGOSS  Panel  provides  the  overall  direction  for  organizing,  managing, 
directing,  and  administering  the  IGOSS  Testing  Program.  The  Executive  Director 
serves  at  the  pleasure  of  the  IGOSS  Panel  under  the  terms  of  the  Charter  of  the  IGOSS 
Panel. 

The  IGOSS  Panel,  under  the  direction  of  the  Executive  Director  of  the  IGOSS  Panel, 
has  the  authority  to: 

1)  Establish  and  maintain  the  IGOSS  conformance  testing  program  policies  and 
procedures; 

2)  Register  the  test  methods  used  in  determining  conformance  of  products  to 
IGOSS; 

3)  Develop  and  maintain  the  procedures  to  be  followed  by  clients  of  accredited  test 
laboratories  in  order  to  attain  product  registration; 

4)  Issue  a statement  of  Registration  based  on  the  results  of  a test  report; 

5)  Establish  the  accreditation  criteria  for  OSI  test  laboratories; 

6)  Maintain  and  periodically  publish  a register  of  products  that  have  passed 
conformance  testing,  and  a register  of  products  that  have  passed  interoperability 
testing. 

7)  Coordinate  with  other  assessment,  accreditation  and  certification  authorities  for 
the  purpose  of  harmonizing  methods  and  making  provisions  for  mutual 
recognition  of  conformance  testing  results; 

8)  Evaluate  and  resolve  disputes  on  all  matters  concerning  conformance  testing  for 
IGOSS; 

9)  Periodically  assess  the  need  for  a conformance  testing  program,  maintain  test 
method  assessment  and  test  laboratory  accreditation  programs  for  IGOSS; 

10)  Maintain  and  publish  a register  of  accredited  test  laboratories  to  perform  IGOSS 
conformance  testing; 

11)  Maintain  and  publish  a register  of  accredited  test  laboratories  recognized  by  the 
IGOSS  Panel  CSLto  perform  IGOSS  Means  of  Testing  Validation; 
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12)  Establish  the  fees  or  rates  for  the  IGOSS  Panel's  Agent  provided  products  and 
services;  and 

13)  Announce  in  the  Federal  Register  and/or  the  Commerce  Business  Daily  the 
availability  of  an  assessment  service. 

NIST/CSL  is  the  permanent  Agent  of  the  IGOSS  Panel.  NIST/CSL  is  responsible  for 
defining  procedures  and  conducting  the  program  of  registration  and  assessments. 
Since  the  IGOSS  Testing  program  is  built  from  the  U.S.  GOSIP  Testing  Program, 
NIST/CSL  will  continue  to  use  the  services  of  the  Joint  Interoperability  Test  Center 
(JITC).  In  the  remaining  part  of  this  document,  the  term  "Agent  of  the  IGOSS  Panel*"  Is 
used  to  Identify  NIST/CSL  and/or  JITC. 

2.2  IGOSS  Accreditation  Authority 

NVLAP  and  SCC  are  the  Accreditation  Authorities  for  IGOSS  testing.  The  role  of 
NVLAP  and  SCC  is  to  assess  and  accredit  testing  laboratories,  using  the  laboratory 
accreditation  procedures  provided  for  OSI  Testing  [NVLAP  1-2]  or  [SCC.  1-6] 

2.3  Suppliers  of  the  Means  of  Testing 

Suppliers  undertake  to  develop  and  supply  to  test  laboratories  the  means  of  testing. 
Suppliers  may  be  commercial,  governmental,  educational,  or  foreign-based 
organizations. 

The  responsibilities  of  a supplier  are  to: 

1)  Develop  the  means  of  testing  for  IGOSS  protocols  in  accordance  with  the  criteria 
specified  In  this  report; 

2)  Undertake  to  maintain  the  means  of  testing  to  reflect  changes  in  the  published 
standards  and  workshop  agreements; 

3)  Obtain  and  maintain  assessment  and  registration  of  their  product(s); 

4)  Pay  all  relevant  fees. 

2.4  Conformance  Test  Laboratories 

Test  laboratories  perform  conformance  testing  in  accordance  with  IGOSS  Panel 
approved  procedures.  Test  laboratories  may  be  commercial  laboratories  (third-party), 
vendor  laboratories  (first-party),  U.S  or  foreign-based  laboratories. 

Only  test  laboratories  accredited  under  an  IGOSS  Panel  approved  laboratory 
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accreditation  program,  or  test  laboratories  established  as  a result  of  mutual  recognition 
arrangements  with  NVLAP  shall  be  recognized  by  IGOSS  Panel  to  do  IGOSS 
conformance  testing. 

The  responsibilities  of  a test  laboratory  are  to: 

1)  Obtain  and  maintain  laboratory  accreditation  as  appropriate; 

2)  Conduct  conformance  testing  In  accordance  with  the  IGOSS  Panel  prescribed 
procedures; 

3)  Prepare  SCTR  and  PCTRs  in  accordance  with  the  IGOSS  Panel  prescribed 
procedures  as  a result  of  the  testing  performed; 

4)  Participate  in  proficiency  testing  as  required; 

5)  Pay  all  relevant  fees; 

6)  Participate  in  training  sessions  or  meetings  as  required  by  the  IGOSS  Panel  to 
remain  up-to-date  on  changes  to  the  conformance  testing  procedures; 

7)  Provide  feedback  to  the  IGOSS  Panel  on  problems  and  improvements  relating  to 
the  conformance  testing  procedures; 

2.5  Clients 

Clients  are  responsible  for  submitting  requests  for  product  conformance  testing  to  an 
accredited  test  laboratory  in  accordance  with  testing  laboratory  prescribed  procedures. 
The  responsibilities  of  a client  include: 

1)  Provide  complete  and  accurate  information  to  the  test  laboratory  for  the 
performance  of  the  requested  conformance  testing; 

2)  Unless  otherwise  agreed  to  by  the  test  laboratory,  provide  the  test  facilities  and 
materials  necessary  for  testing; 

3)  Provide  a product  conforming  with  IGOSS; 

4)  Provide  copies  of  the  PCTR,  SCTR,  and  all  other  required  documentation  to  the 
registration  authority,  after  successful  conformance  testing  by  an  accredited  test 
laboratory,  for  the  purpose  of  registration. 

2.6  Criteria  for  Registration  of  a Conformant  IGOSS  Product 

1)  Submit  to  Agent  of  the  IGOSS  Panel*  a PICS  and  PIXIT  for  a product  claiming  to 

conform  to  IGOSS  specifications; 
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2)  Provide  Protocol  Conformance  Test  Report(s)  and  System  Conformance  Test 
Report  for  the  product; 

3)  These  reports  are  to  be  produced  after  successful  conformance  testing  by  an 
accredited  test  laboratory  using  a registered  means  of  testing,  including 
registered  abstract  test  suites. 

4)  Pay  the  appropriate  registration  fees. 

Annex  A presents  comprehensive  information  for  IGOSS  Product  Registration. 

3.  ACCREDITATION 

The  Agent  of  the  IGOSS  Panel*  will  carry  out  Its  responsibilities  for  conformance  testing 
through  test  laboratories  judged  to  be  competent  to  objectively  perform  the  necessary 
tests.  Laboratory  accreditation  serves  as  a basis  for  determining  laboratory 
competence.  The  purpose  is  to  insure  that  testing  facilities  are  available  for  obtaining 
an  unbiased  assessment  of  products  regarding  IGOSS  conformance. 

To  ensure  that  test  laboratories  are  using  tools  which  are  capable  of  performing 
accurate  and  adequate  assessments,  the  IGOSS  Panel  defines  in  clause  4 the 
requirements  upon  each  party  involved. 

Wherever  appropriate  for  a given  IGOSS  protocol,  the  IGOSS  Panel  will  draw  upon  the 
NVLAP  and  SCC  as  the  basis  for  accrediting  test  laboratories.  The  Agent  of  the  IGOSS 
Panel*  shall  establish  technical  criteria  for  laboratory  accreditation.  Technical  experts 
for  assessing  laboratory  competence  (assessors)  may  be  drawn  from  qualified 
Government,  academic,  industrial,  or  independent  organizations. 

The  objectives  of  laboratory  accreditation  are  to: 

1)  Identify  technically  competent  testing  services; 

2)  Assess  and  evaluate  each  test  laboratory  accredited  to  do  testing  for 
conformance  to  IGOSS  by: 

a)  Conducting  periodic  laboratory  proficiency  testing  to  identify  testing 
capability, 

b)  Initially,  and  periodically  thereafter,  conducting  on-site  assessments  to 
determine  compliance  with  the  accreditation  criteria,  and 

c)  Conducting  visits  to  verify  reported  changes  in  the  laboratory's  personnel, 
facilities,  and  operations,  or  to  explore  possible  reasons  for  poor 
performance  in  testing  practices; 
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3)  Insure  that  the  test  laboratory  has  adequate  quality  control,  facilities,  equipment 
and  personnel  to  conduct  testing; 

4)  Determine  that  the  test  laboratory  staff  is  adequately  trained  in  using  the 
appropriate  registered  Means  of  Testing,  following  the  prescribed  conformance 
testing  procedures; 

5)  Insure  that  adequate  records  are  maintained  to  support  the  testing  performed 
and  that  test  reports  are  produced  to  provide  the  necessary  information  for 
determining  conformance  to  IGOSS; 

6)  Notify  the  test  laboratory  of  deficiencies; 

7)  Establish  criteria  and  procedures  for  test  laboratories  to  both  obtain  and  maintain 
accreditation. 


4.  Means  of  Testing  Assessment 

Prerequisites  of  the  methodology  defined  below  are  that  an  abstract  test  suite  (ATS) 
exists,  standardized  or  not,  which  has  been  publicly  reviewed,  with  respect  to  the 
IGOSS  PICS,  amended  as  necessary,  and  is  registered  by  the  Agent  of  the  IGOSS 
Panel*. 


4.1  Registered  Abstract  Test  Suites 

A recognized  abstract  test  suite  shall  be  registered  by  the  Agent  of  the  IGOSS  Panel* 

(hereafter  called  a registered  ATS)  upon  approval  from  the  OIW  Conformance  Testing 

SIG. 

Amendment  of  an  ATS  is  defined  to  be: 

1)  delete  test  cases  which  are  not  applicable  to  a profile; 

2)  specify  constraints  based  upon  agreements  of  the  OIW  which  may  be  made 
either  more  rigorous  or  less  rigorous; 

3)  specify  additional  test  cases  as  necessary  to  encompass  all  mandatory  and 
optional  features  using  criteria  stated  In  Part  ll  of  this  Report; 

4)  Registration  of  the  ATS  shall  be  staged  to  accommodate  Improvements  in  the 
state  of  the  art  of  test  suite  development.  Each  ATS  will  be  harmonized  with  the 
ISO  work  and  will  reach  stability  with  the  International  Standard. 
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4.2  Means  of  Testing  Supplier 

Suppliers  of  a Means  of  Testing  shall: 

1)  Request  assessment  of  a product  by  submitting  the  forms  and  fees  as  required 
by  an  approved  MOT  assessment  laboratory; 

2)  Identify  the  mapping  between  each  abstract  test  case  and  the  supplier's 
realization  of  it; 

3)  Arrange  for  a mutually  satisfactory  date  for  assessment  of  the  MOT ; 

4)  Demonstrate  to  the  satisfaction  of  the  MOT  Assessment  and  Registration 
Authority,  that  the  executable  test  cases  achieve  the  test  purposes  and  exhibit 
the  dynamic  behavior  specified  In  the  ATS  when  executed  against  an 
implementation  of  the  corresponding  OSI  protocol; 

5)  Have  a quality  management  program  that  assures  maintenance  and 
convergence  of  their  product(s)  such  that  the  product(s)  meet  the  requirements 
of  registered  ATS(s)  and  the  Means  of  Testing  Assessment  and  Registration 
Authority; 

6)  Have  adequate  mechanisms  for  distribution  of  updates  and  corrections  to  test 
laboratories  employing  the  supplier  products,  in  accordance  with  the  Agent  of  the 
IGOSS  Panel*  staged  improvement  procedures,  and  mechanisms  to  notify  test 
laboratories  of  known  problems  in  products. 


4.3  Approved  MOT  Assessment  Laboratory 

An  Approved  MOT  Assessment  Laboratory  shall: 

1)  Upon  receipt  of  a request  from  an  MOT  supplier,  and  required  fees,  arrange  for 
a mutually  acceptable  date  for  assessing  the  product; 

2)  Select  one  or  more  assessors  from  a group  of  experts  who  have  no  vested 
Interest  In  the  product  and  are  neutral  with  respect  to  the  results  of  the 
assessment,  are  technically  competent  with  respect  to  the  OSI  protocol(s), 
IGOSS  requirements,  and  the  conformance  test  methodology  employed; 

3)  Assess  the  Means  of  Testing  using  either  a reference  implementation,  or  an 
otherwise  available  implementation  of  the  protocol(s)  and  following  the 
procedures  and  criteria  defined  in  Annex  B and  [JITC  1]. 

4)  Upon  completion  of  the  Means  of  Testing  Assessment  provide  the  Means  of 
Testing  Test  Report  to  the  Registration  Authority. 
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4.4  MOT  Registration  Authority 

The  Agent  of  the  IGOSS  Panel*  is  the  MOT  registration  authority.  The  Agent  of  the 
IGOSS  Panel*  has  the  authority  to  approve  specific  organizations  that  implements 
procedures  and  criteria  equivalent  to  the  ones  defined  in  Annex  B to  participate  in  the 
IGOSS  MOT  Assessment. 

4.5  Role  of  the  MOT  Supplier 

Using  the  test  cases  selected,  the  supplier  shall: 

1)  provide  the  assessor  with  the  facility  to  select  and  execute  test  cases  on  the 
MOT  under  assessment; 

2)  execute  test  cases  selected  by  the  assessor(s); 

3)  produce  log  files  showing  detailed  protocol  exchange  behavior; 

4)  make  these  log  files  available  to  the  assessor(s)  for  their  analysis. 

When  the  MOT  has  been  registered,  the  MOT  Supplier  must  provide  a non-altered 
copy  of  the  MOT  Qualification  Report  to  all  its  NVLAP  and  SCC  Clients. 

5.  REGISTERS  EMPLOYED 

Essential  to  the  operation  of  the  provisions  of  this  report  are  registers  maintained  by  the 
Agent  of  the  IGOSS  Panel*.  The  names  of  the  registers  and  a brief  description  of  each 
are  below. 

PICS  Proformae  for  IGOSS 

For  each  IGOSS  protocol  a PICS-Proforma  will  be  identified  and  registered  . PICS- 
Proforma  are  used  by  OSI  Product  Vendors  to  make  a statement  of  an  OSI 
implementation,  or  system,  stating  which  capabilities  and  options  have  been 
implemented,  for  a given  OSI  protocol. 

Abstract  Test  Suites  for  IGOSS 

For  each  IGOSS  protocol  a test  suite  composed  of  test  purposes  or  abstract  test  cases 
is  placed  in  the  public  domain  and  designated  as  the  Registered  Abstract  Test  Suite. 
The  tests  are  updated  from  time  to  time  to  harmonize  with  International  Standard  test 
suites. 

Assessed  Means  of  Testing  for  IGOSS 

For  IGOSS  profiles  or  substacks,  the  means  of  testing  are  assessed  according  to  the 
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criteria  published  in  Annex  B and  in  [JITC-1].  The  treatment  of  derived  implementations 
of  a MOT  is  described  in  the  documents.  Any  qualified  system  may  be  registered  for 
use  in  IGOSS  conformance  testing. 

Approved  Laboratories  for  Means  of  Testing  Assessment 

An  accredited  laboratory  that  has  been  approved  by  the  Agent  of  the  IGOSS  Panel*  to 
conduct  means  of  testing  assessment.  All  the  approved  laboratories  for  means  of 
testing  assessment  follow  the  procedures  describes  in  Annex  B. 

Laboratories  Accredited  for  IGOSS  Conformance  Testing 

Any  testing  laboratory  which  complies  with  the  provisions  of  this  Special  Publication  and 
a companion  handbook  [NIST  16],  as  administered  by  the  National  Voluntary 
Laboratory  Accreditation  Program,  may  be  accredited  and  added  to  the  register. 
Compliance  includes  the  operation  of  a registered  Means  of  Testing  realizing  one  or 
more  registered  Abstract  Test  Suites. 

Conformance  Tested  IGOSS  Products 

IGOSS  products  which  have  been  successfully  tested  by  an  accredited  test  laboratory 
using  a registered  Means  of  Testing,  including  registered  Abstract  Test  Suites,  may  be 
added  to  the  register.  The  treatment  of  derived  implementations  is  described  in  clause 
7.2.3  below  and  Annex  C.  Addition  to  this  register  is  required  before  IGOSS 
interoperability  testing  is  conducted. 

Interoperability  Test  Suites  for  OSI  Products 

For  each  IGOSS  application  a test  suite  composed  of  test  purposes  is  designated  as 
the  Registered  Interoperability  Test  Suite.  The  tests  are  updated  from  time  to  time  to 
achieve  international  harmonization. 

Reference  Entities  for  Means  of  Testing  Assessment 

For  each  IGOSS  application  or  intermediate  system  and  its  supporting  stack,  one  of 
which  has  been  selected  to  support  the  Means  of  Testing  assessment. 

Interoperability  Test  and  Registration  Services 

Any  organization  may  offer  to  define  procedures  for  the  conduct  of  multi-vendor 
interoperability  testing  and  to  register  the  results  of  testing.  Any  such  organization 
which  is  approved  by  the  IGOSS  Panel*  is  entered  onto  this  meta-register.  The  number 
of  IGOSS  approved  Interoperability  Services  is  not  limited.  General  criteria  for  approval 
are  given  in  Section  7. 3. 2. 2. 
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6.  EMPLOYMENT  OF  THE  OSI  CONFORMANCE  TESTING  METHODOLOGY 

ISO's  OSI  Conformance  Testing  Methodology  and  Framework  [ISO  1 13]  is  an  evolving 
multi-part  international  standard.  It  defines  terminology,  concepts,  and  requirements 
for:  (1)  other  standards  bodies  who  are  responsible  for  producing  abstract  test  suites; 
(2)  suppliers  of  a means  of  testing  (real  test  systems);  (3)  test  laboratories;  (4)  clients  of 
test  laboratories;  (5)  information  exchanged  between  test  laboratories  and  their  clients; 
and  (6)  proformae  for  test  reports  (whose  content  is  outlined  In  Parts  4 and  5 of  the  ISO 
Conformance  Methodology  [ISO  113]  and  is  detailed  in  standards  associated  with 
abstract  test  suites). 

This  standard  provides  the  basis  for  the  conformance  aspects  of  the  IGOSS  testing 
program;  In  certain  instances,  however  the  work  of  ISO  is  not  applicable  to  this  report. 
For  example,  when: 

1)  Standardized  abstract  test  suites  do  not  exist  for  an  IGOSS  protocol, 

2)  Standardized  abstract  test  suites  do  not  test  for  features  mandated  by  IGOSS, 

3)  ISO  Conformance  methodology  does  not  address  physical  media,  and 

4)  Interoperability  testing  is  required. 

The  means  of  testing  employed  by  accredited  test  laboratories  may  be  a product  of 
either  the  public  sector  or  private  sector,  if  the  latter  is  a commercially  available 
product. 

This  document  references  abstract  test  suites  (or  test  methodology)  for  IGOSS  which 
have  been  approved  by  the  OSE  Implementors'  Workshop.  They  may  be  produced  by 
the  public  sector,  including  standards  bodies,  or  the  private  sector.  Any  registered 
abstract  test  suite  or  other  test  methodology  employed  shall  be  in  the  public  domain, 
without  protection  of  copyright. 

This  document  provides  criteria  for  assessment  of  the  coverage  and  quality  of  test 
suites.  As  standardized  abstract  test  suites,  and  their  derived  executable  test  suites, 
become  available,  they  will  be  assessed  for  their  adequacy  under  the  criteria  in  this 
document,  in  order  that  they  may  be  approved  for  use  in  testing  IGOSS  products. 

Staged  improvements  to  the  abstract  test  suites  will  be  conducted.  The  intention  Is 
ultimately  to  harmonize  with  International  Standard  OSI  test  suites. 


7.  TESTING  FRAMEWORK 

7.1  Relation  Between  Testing  Phases 

The  Immediate  goal  of  testing  communications  products  which  claim  to  conform  to  the 
IGOSS  specifications  Is  to  qualify  them  for  inclusion  in  either  the  Register  of 
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Conformance  Tested  IGOSS  Products,  one  of  the  Register  of  Interoperability  and 
Registration  Services,  or  both.  The  phases  of  testing  coinciding  with  registration  are 
conformance  testing  and  interoperability  testing,  respectively. 

1)  Successful  conformance  testing  by  an  accredited  test  laboratory  using  a 
registered  Means  of  Testing,  Including  registered  Abstract  Test  Suites,  leads  to 
addition  to  the  Register  of  Conformance  Tested  IGOSS  Products. 

2)  Successful  interoperability  testing  with  other  compatible  products  using  a 
recognized  methodology,  that  is,  one  on  the  Register  of  Interoperability  Test  and 
Registration  Services,  leads  to  publicly  accessible  documentation  of  pair-wise 
multi-vendor  Interoperability. 

Requirements  for  IGOSS  product  suppliers  to  enter  each  phase  of  testing  follow. 
Conformance 

1)  Develop  a IGOSS  conformant  product,  or  a partial  stack  thereof,  which  is 
testable  using  at  least  one  of  the  methods  specified  here  in. 

2)  Provide  a PICS  to  an  accredited  test  laboratory  specifying  functionality 
supported  in  the  implementation  for  each  protocol  in  the  stack. 

3)  Provide  a PIXIT  to  the  accredited  test  laboratory  for  the  stack/substack. 

4)  For  stacks  which  build  on  substacks  which  have  been  previously  conformance 
tested  within  the  IGOSS  testing  process,  provide  an  SCTR,  PCTR  and  evidence 
of  Registration  for  the  previously  tested  functionality.  For  Instance  if  the  Session 
protocol  is  to  be  tested  over  transport  class  4 protocol  (TP4),  an  SCTR,  PCTR 
and  statement  of  registration  should  be  furnished  for  the  TP4  substack,  as 
evidence  that  all  the  supporting  protocols  do  not  need  to  be  completely  re- 
settled. 

Interoperability  Testing 

Entry  on  the  Agent  of  the  IGOSS  Panel*  register  of  conforming  IGOSS  products  Is  a 
prerequisite  to  IGOSS  Interoperability  testing. 

Detailed  mechanisms  applicable  to  conformance  and  interoperability  testing  and  the 
assessment  thereof,  are  given  in  the  following  clauses. 
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7.2  Conformance  Testing 
7.2.1  What  Is  To  Be  Tested 

Products  of  the  following  types  may  be  made  available  for  conformance  testing: 

• 7 layer  Application  stacks, 

• 7 layer  Relay  stacks, 

• 5-layer  Session  Stacks, 

• 4 layer  transport  stacks, 

• 2 or  3 layer  Network  stacks, 

• 3 layer  Intermediate  System  stacks. 

This  structuring  explicitly  recognizes  test  platforms  (substacks)  employed  by  existing 
testing  technology. 

In  conformance  testing,  previously  tested  End  System  substacks  may  be  carried 
forward  Into  larger  substacks.  In  such  cases,  comprehensive  re-testing  of  previously 
tested  functionality  is  not  necessary:  Basic  Interconnection  Testing  is  sufficient, 
providing  that  Protocol  Conformance  Test  Reports  are  furnished  for  previously  tested 
protocols.  However,  the  same  principle  does  not  work  in  reverse.  If  a 7 layer  stack  is 
tested,  using  Single-layer  embedded  methods  for  the  lower  layers,  and  a substack  is 
subsequently  extracted  and  added  as  a component  of  a different  7 layer  stack,  then  the 
whole  of  the  new  stack  shall  be  comprehensively  tested.  This  Is  because  embedded 
testing  alone  does  not  provide  sufficient  confidence  in  a lower  layer  protocol  when 
considered  outside  of  Its  original  stack  context. 

Example: 

FTAM/ACSE/Presentation/Sesslon/TP4/CLNP/LLC1/8802.3 
Conformance  testing  of  substacks  may  proceed  as  follows. 

1)  The  subnetwork  protocols  (LLC1/8802.3)  may  be  tested  (although  this  is  not 
separately  mandated)  and  a System  Conformance  Test  Report  (SCTR)  is 
produced  and  statement  of  registration  may  be  produced. 

2)  TP4/CLNP  Is  offered  for  testing  over  802.3;  the  statement  of  registration  is 
provided  to  demonstrate  conformance  to  8802.3.  Basic  Interconnection  testing 
is  conducted  to  establish  the  basic  workability  of  the  substack.  CLNP  is  tested 
by  coordinated  single-layer  embedded  means;  TP4  is  tested  by  coordinated 
single  layer  means.  If  conformance  to  both  protocols  is  established,  a second 
SCTR  is  produced  for  the  substack. 
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3)  FTAM  is  offered  for  testing  over  TP4;  the  second  statement  of  registration  is 
provided  to  demonstrate  conformance  to  TP4,  CLNP  and  8802.3.  Basic 
interconnection  testing  is  conducted  to  establish  the  basic  workability  of  the 
stack  (as  an  FTAM  Initiator  or  as  an  FTAM  Responder).  Session,  presentation 
and  ACSE  are  tested  by  remote  single-layer  embedded  means  for  use  with 
FTAM  Responders,  and  by  distributed  single-layer  embedded  means  for  use 
with  FTAM  Initiators.  An  FTAM  Responder  is  tested  by  remote  single-layer 
means;  an  FTAM  Initiator  is  tested  by  distributed  single-layer  means.  If 
conformance  to  all  protocols  Is  established,  a statement  of  registration  is 
produced  for  the  FTAM  Initiator  and  Responder  stacks. 

7.2.2  How  Testing  Is  Conducted 

These  IGOSS  Testing  Guidelines  are  guided  by  the  recommendations  given  In  the  OSI 
Conformance  Testing  Methodology  and  Framework  [ISO  113].  MOT  suppliers  and 
conformance  test  laboratories  are  expected  to  be  familiar  with  Part  5 which  provides 
Requirements  on  Test  Laboratories  and  Clients  for  the  Conformance  Assessment 
Process,  and  with  the  General  Principles  and  Abstract  Test  Methods  defined  In  Parts  1 
and  2. 

7.2.2. 1 Testing  Elements 

For  each  stack  supplied,  the  product  configuration  determines  the  test  method  used. 
Part  1 defines  Abstract  Test  Methods  which  may  be  employed  and  provides  guidance 
on  their  applicability  to  real  test  systems.  Central  to  the  concept  of  conformance  testing 
is  the  ability  to  control  and  observe  events  of  the  Implementation  Under  Test  (lUT) 
within  the  System  Under  Test  (SUT).  Every  test  method  has,  at  minimum,  a Point  of 
Control  and  Observation  (PCO)  through  the  medium  which  connects  the  means  of 
testing  with  the  SUT.  This  is  the  point  at  which  the  means  of  testing  injects  valid  and 
invalid  Protocol  Data  Units  (PDUs)  of  the  protocol  or  protocols  under  test,  and  observes 
PDUs  returned  from  the  SUT.  Any  SUT  which  Is  accessible  only  through  this  PCO  is 
testable  using  the  Remote  method.  The  distributed  and  coordinated  methods  provide 
extra  control  and  coordination  with  the  SUT,  and  this  is  usually  effected  through  test 
coordination  procedures  which  exist  between  the  means  of  testing  and  the  SUT.  In  all 
cases  the  Initial  stimulus  for  testing  comes  from  the  means  of  testing.  For  the  remote 
method,  the  SUT  Is  stimulated  by  protocol  data  units  received  via  an  underlying  OSI 
service.  For  the  distributed  method,  the  SUT  Is  stimulated  directly  at  the  (N)  service,  by 
coordination  interactions  between  the  Means  of  Testing  and  the  SUT.  For  the 
coordinated  method  the  SUT  is  stimulated  to  generate  (N)-PDUs  as  a result  of  prior 
Interactions  between  the  lower  tester  and  upper  tester,  by  means  of  a test  management 
protocol. 

The  OSI  Conformance  Testing  Methodology  and  Framework  structures  a test  suite  into 
Basic  Interconnection,  Capability  and  Behavior  tests.  The  first  two  of  these  categories 
are  proper  subsets  of  the  third  (although  the  test  purposes  may  be  different).  In  the 
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limit,  Behavior  tests  provide  exhaustive  coverage  - a limit  which  is  by  no  means 
practical.  Basic  Interconnection  tests  are  used  in  practical  testing  situations  to  check 
out  the  basic  operation  of  the  linkage  between  the  SUT  and  the  means  of  testing,  and 
as  such  are  not  mandatory  for  coverage  purposes.  Capability  tests  are  intended  to 
provide  100  per  cent  'breadth'  of  test  coverage,  i.e.,  at  least  one  test  per  function 
specified  in  the  protocol/Stable  Implementor's  Agreements  for  all  mandatory  and 
optional  features.  Behavior  tests  provide  extra  depth  of  coverage  over  all  of  the 
capabilities.  In  regard  to  coverage,  options  In  a protocol  are  not  optional  in  a test  suite. 
Tests  must  be  available  for  each  option,  even  though  they  may  not  be  selected  for  use 
with  a particular  I LIT. 

The  administration  of  IGOSS  testing  Includes  recognition  and  registration  of  abstract 
test  suites  for  each  IGOSS  protocol.  Specific  provisions  are  given  In  the  Annex  B. 

7. 2.2.2  The  Process 

The  conformance  assessment  process  Includes: 

Preparation  for  testing. 

Test  operations. 

Test  report  production. 

The  following  clauses  provide  a brief  description  of  these  phases.  A comprehensive 
description  Is  given  In  Part  5 of  IS  9646.  Accreditation  of  conformance  testing 
laboratories  is  directly  based  on  the  use  of  that  test  methodology. 

7. 2.2.3  Preparation  For  Testing 

The  preparation  phase  includes  general  documentation  and  configuration  steps  which 
must  be  carried  out  prior  to  conducting  a test  campaign.  This  includes  furnishing  of 
IGOSS  PICS,  PIXIT,  and  any  vendor  configuration  requirements  to  a test  laboratory. 

7.2.2.4  Test  Operations 

The  test  operations  phase  includes  static  conformance  assessment,  test  selection  and 
parameterization,  followed  by  dynamic  testing.  Test  selection  is  based  on  options 
claimed  to  be  supported  in  the  lUT,  as  documented  in  the  IGOSS  PICS. 
Parameterization  of  selected  tests  is  based  on  information  provided  In  the  PIXIT. 
Dynamic  testing  occurs  using  the  executable  realization  of  the  selected  abstract  test 
cases.  In  which  each  test  case  is  executed  against  the  lUT  to  produce  a Verdict.  Any 
'Inconclusive'  verdicts  may  be  resolved  into  'Pass'  or  'Fail'  at  this  time. 

7.2.2.5  Test  Report  Production 

Test  report  production  is  a phase  of  assessment  of  the  results  of  dynamic  testing,  and 
production  of  System  Conformance  Test  Report  and  Protocol  Conformance  Test 
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Report(s),  recording  the  verdicts  determined  during  the  dynamic  assessment. 

7.2.2.6  Addressing 

The  testing  of  addressing  includes  a static  check  against  the  PIXIT  for  employment  of 
the  IGOSS  addressing  structure,  and  Basic  Interconnection  Tests  to  ensure  that 
addressing  is  accurate. 

7. 2.2.7  Evaluation  of  Conformance  Testing 

Detailed  criteria  for  the  evaluation  of  conformance  testing  are  provided  by  the  Annex  B 
for  test  systems;  [NIST^16],  [NIST  26]  and  [SCC  1]  for  conformance  testing  laboratories. 

7.2.3  Treatment  of  Derived  Products 

In  certain  circumstances  a Product  Registration  may  be  derived  from  a tested  'base' 
Implementation,  without  requiring  further  formal  testing  (see  Annex  A,  A.  1.2  and  A.  1.3). 
The  following  guidelines  are  applicable: 

(1)  In  certain  circumstances  a IGOSS  Product  may  be  derived  from  a tested  'base' 
implementation,  without  requiring  complete  formal  testing.  In  his  application  for 
derived  product  registration  the  vendor  must  provide  the  rationale  for  his 
assertion  that  the  Implementation  is  substantially  unaltered  (from  the  originally 
tested  base  product).  When  considering  registration  requests  for  derived 
products  the  Agent  of  the  IGOSS  Panel*  will  use  the  following  guidelines. 

(a)  The  host  and  target  computer  systems  of  the  base  and  derived  IGOSS 
Implementations  have  compatible  instruction  sets  and  operating  systems. 
Common  examples  of  compatible  instruction  sets  and  operating  systems 
are  two  different  computer  system  models  in  a manufacturer's  product  line 
or  the  computer  systems  produced  by  different  manufacturers  that  use 
the  same  hardware  mechanisms  and  operating  systems. 

(b)  The  IGOSS  Implementation  proposed  for  registration  was  derived  from 
the  base  implementation  by  changes  that  are  within  the  scope  of 
accepted  software  maintenance  practices.  Arguments  along  this  line 
should  be  included,  in  writing,  with  the  application. 

(c)  Basic  Interconnection  Testing  is  conducted  on  the  derived 
implementation,  which  results  in  the  production  of  test  logs  showing  PDU 
exchange  activity  which  is  the  same  as  the  base  implementation  or.  If 
there  are  minor  differences,  these  differences  are  justified  as  being  within 
the  scope  of  accepted  software  maintenance  practices.  Samples  of  the 
test  logs  and  associated  PCTRs  for  the  basic  interconnection  tests  should 
support  the  derived  product  application. 
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(d)  The  base  implementation  was  assessed  in  accordance  with  the  Agent  of 
the  IGOSS  Panel*  procedures  and  is  registered  with  the  Agent  of  the 
IGOSS  Panel*. 


(2)  An  IGOSS  Product  may  be  registered  for  use  over  a different  communications 
platform  than  that  originally  registered,  without  complete  formal  testing.  When 
considering  registration  requests  the  Agent  of  the  IGOSS  Panel*  will  use  the 
following  guidelines. 


(a)  The  base  implementation  was  assessed  In  accordance  with  the  Agent  of 
the  IGOSS  Panel*  procedures  and  is  registered  with  the  Agent  of  the 
IGOSS  Panel*. 


(b)  The  communications  platform  to  which  the  Implementation  is  ported  was 
assessed  in  accordance  with  the  Agent  of  the  IGOSS  Panel*  procedures 
and  is  registered  with  the  Agent  of  the  IGOSS  Panel*. 

(c)  Basic  Interconnection  Testing  is  conducted  on  the  ported  platform  which 
results  In  the  production  of  test  logs  showing  PDU  exchange  activity  which 
is  the  same  as  the  base  implementation  or,  If  there  are  minor  differences, 
these  differences  are  justified  as  being  within  the  scope  of  accepted 
software  maintenance  practices.  Samples  of  the  test  logs  should  support 
the  ported  platform  application. 

A derived  or  ported  Implementation  may  lose  its  registration  if  it  is  challenged 
successfully.  Such  challenges  are  described  in  Section  A.2. 


7.3  Interoperability  Testing 

The  market  and  Industry  have  matured  significantly  over  the  last  couple  of  years  to  a 
point  where  interoperability  testing,  performed  and  managed  according  to  formal 
methodology(ies)  and  in  accordance  with  globally  accepted  practices,  is  seen  more  and 
more  by  major  users  and  vendors  as  today's  true  complement  to  conformance  testing. 
The  objective  here  being  "to  provide  a service  which,  for  a specific  protocol  or  set  of 
protocols,  will  test  whether  two  separate  Implementations  are  able  to  Inter-work  in  a 
variety  of  operational  conditions".  The  growing  Importance  and  understanding  of 
Interoperability  Is  also  increasing. 

7.3.1  What  Is  Tested 

Products  made  available  for  interoperability  testing  may  be: 

7 layer  Application  stacks, 

7 layer  Application  Relay  stacks, 

3 layer  Intermediate  System  stacks. 
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In  the  case  of  End  Systems,  interoperability  testing  could  proceed  by  pair-wise 
operation  of  compatible  IGOSS  systems.  For  Instance,  supplier  A wishing  to  test  an 
FTAM  Initiator  against  supplier  B should  ensure  first  that  B's  product  provides  FTAM 
Responder  capability  with  a compatible  IGOSS  profile.  Moreover  If  B provides  sender 
only,  then  A must  be  capable  of  receiving.  It  Is  for  this  reason  that,  in  the  same  way  as 
with  conformance  testing,  a static  analysis  phase  Is  necessary  to  determine  whether 
testing  can  proceed  at  all. 

Interoperability  testing  with  Intermediate  Systems  (IS)  operates  on  a multi-peer  basis; 
pairs  of  End  Systems  communicate  through  one  or  more  3 layer  Intermediate  Systems. 
An  IS  must  be  capable  of  working  with  each  End  System  on  each  supported 
subnetwork,  and  of  routing  data  between  pairs  of  End  Systems.  In  a complex 
concatenated  network,  an  IS  must  also  route  data  from  and  to  other  ISs.  Interoperability 
testing  with  Message  Transfer  Agent  relay  entities  proceeds  in  a similar  manner  to 
Intermediate  System  testing,  on  a multi-peer  basis. 

7.3.2  How  Testing  Is  Conducted 

The  interoperability  testing  requirements  of  U.S.  IGOSS  are  grounded  in  the  use  of  the 
following  registers: 

the  Register  of  Interoperability  Test  Suites 
the  Register  of  Interoperability  Testing  Services 
the  Register  of  IGOSS  Compliant  products 

7.3.2.1  Requirements  of  an  Interoperability  Testing  Suite 

In  order  to  be  entered  onto  the  register,  an  Interoperability  Test  Suite  must: 

1)  be  freely  available  (i.e.,  public  domain  or  minimum  costs  of  duplicating), 

2)  be  subject  to  public  review, 

3)  be  capable  of  exercising  the  mandatory  and  optional  functionality  of  each  IGOSS 
application, 

4)  describe  the  purpose  of  each  test  and  procedures  by  which  the  test  purpose  can 
be  realized, 

5)  specify  what  constitutes  a 'pass'  or  'successful  execution'  for  each  test,  and, 

6)  be  supported  by  an  organization  recognized  by  the  Agent  of  the  IGOSS  Panel*. 

For  each  IGOSS  application,  only  one  Interoperability  Test  Suite  may  be  registered. 
When  a Test  Suite  becomes  qualified  it  will  be  provisionally  registered  until  the  next 
IGOSS  Version  release.  It  will  be  reviewed  and  updated  at  that  time.  Staged 
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improvements  leading  to  harmonization  proceed  in  this  way  until  the  completed  Test 
Suite  is  fully  registered.  Registration  remains  current  while  the  Implementation 
Agreements  or  the  base  standards  remain  valid.  If  more  than  one  valid  Interoperability 
Test  Suite  becomes  available  for  a particular  application  stack,  then  one  only  will  be 
selected  by  Agent  of  the  IGOSS  Panel*  for  registration. 

7. 3.2.2  Requirements  of  an  Interoperability  Testing  Service 

In  the  same  way  as  with  Conformance  Testing,  Interoperability  Testing  can  be 
characterized  as  having  the  three  phases  of  preparation,  test  operations  and  test  report 
production.  In  order  to  become  registered,  any  forum  seeking  to  offer  an 
Interoperability  Testing  Service  should  meet  the  following  criteria: 

1)  Be  a legal  entity; 

2)  Make  its  Interoperability  Testing  Procedures  available  upon  request  to  any 
user/supplier; 

3)  Use  (at  the  minimum)  a registered  Interoperability  Test  Suite; 

4)  Conduct  a static  analysis  phase  which  Involves  the  selection  of  a common 
subset  of  the  IGOSS  tests  including  all  of  the  mandatory  tests  and  all  the 
relevant  optional  tests  resulting  from  the  implementation  of  the  same  optional 
functionality; 

5)  Conduct  a dynamic  analysis  phase  in  which  both  IGOSS  product  suppliers  are  In 
agreement  concerning  the  outcome  of  each  test.  At  the  discretion  of  one  or  more 
of  the  IGOSS  product  suppliers,  an  Interoperability  Testing  campaign  may  be 
terminated  before  the  Test  Report  is  produced; 

6)  Issue  a test  report  which  identifies  each  IGOSS  product  supplier,  describes  the 
product  including  the  supporting  stack  of  protocols,  and  provides  the  list  of  all  the 
possible  test  cases  and  identifies  the  tests  selected  and  executed  with  a verdict 
for  each  test.  The  verdict  may  be  'pass'  or  'fall'.  Any  fall  verdict  must  be 
accompanied  by  an  explanation  outlining  the  cause  of  failure.  Any  non  selected 
test  case  must  be  accompanied  by  an  explanation  outlining  the  rational  for  de- 
selection. 

7)  Provide  all  the  relevant  Information  about  the  status  of  compliance  for  the 
product(s).  If  the  product  has  passed  conformance  testing  this  Information  shall 
include  the  accredited  laboratory  where  conformance  testing  took  place,  test 
campaign  date  and  date  of  registration  In  the  IGOSS  Compliant  Product 
Register. 

8)  At  the  request  of  the  Agent  of  the  IGOSS  Panel*,  a copy  of  the  test  report 
resulting  from  any  bilateral  or  multilateral  agreement  made  under  the  auspices  of 
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the  Interoperability  Testing  Service.  A nominal  fee  may  be  charged  for  each 
report  supplied.  Each  IGOSS  product  supplier  may  require  limitations  on  the  use 
for  which  a test  report  is  employed. 

In  cases  where  IGOSS  users  find  a persistent  lack  of  interoperability  among  products 

registered  as  interoperable,  appeals  may  be  made  as  follows: 

1)  To  the  vendor,  or  vendors  of  the  Inoperable  products,  who  shall  make  every 
effort  to  make  good  on  their  warranty. 

2)  To  the  Interoperability  Test  and  Registration  Service  who,  after  investigation  may 
remove  the  product  pair  from  the  register. 

3)  To  the  Agent  of  the  IGOSS  Panel*  who  may  require  to  witness  testing  of  either 
or  both  of  the  products  involved,  under  the  auspices  of  the  Interoperability  Test 
and  Registration  Service,  in  the  original  pairing,  or  in  any  other  pairings  required 
by  the  Agent  of  the  IGOSS  Panel*.  In  the  event  that  the  Agent  of  the  IGOSS 
Panel*  and  the  procuring  Authority  remain  unsatisfied  then  the  Interoperability 
Test  and  Registration  Service  may  have  Its  registration  revoked. 

7.3.3  Treatment  of  Derived  Implementations 

For  the  purpose  of  interoperability  testing,  no  distinction  shall  be  made  for  derived 

Implementations.  Any  product  registered  should  be  subject  to  pairwise  testing. 


8.  RECOGNITION  OF  OTHER  CONFORMANCE  TESTING  ACTIVITIES 

The  IGOSS  Partners  seek  to  provide  economical  and  adequate  conformance  testing.  It 
is  not  the  intent  of  the  IGOSS  Partners  to  duplicate  conformance  testing  activities 
where  those  activities  meet  the  Industry/Government  requirements.  Thus,  the  Agent  of 
the  IGOSS  Panel*  will  coordinate  with  other  organizations  to  harmonize  conformance 
testing  requirements. 

In  meeting  these  objectives,  the  IGOSS  Partners  will  consider  the  use  of  existing  test 
methods,  conformance  testing  procedures,  test  laboratories  and  certification  systems. 

Possible  recognition  of  other  activities  include: 

1)  Foreign-based  test  laboratory  accreditation  and  services, 

2)  Test  method  administration  (maintenance  and  distribution  systems), 

3)  Conformance  testing  procedures, 

4)  Test  reports. 
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5)  Certificates,  and 

6)  Test  method  research  and  development. 

The  Agent  of  the  IGOSS  Panel*  may  use  any  of  the  following  methods  for  formally 
recognizing  the  conformance  testing  activities  of  other  organizations: 

1)  Contract, 

2)  Accreditation, 

3)  Memorandum  of  Understanding,  or 

4)  International  Standard. 

Any  of  the  above  methods  are  acceptable  if  they  do  not  conflict  with  or  compromise  the 
Agent  of  the  IGOSS  Paner's  authority  in  carrying  out  its  responsibilities  or  violate 
Federal  regulations. 

Unless  otherwise  approved  by  the  Agent  of  the  IGOSS  Panel*,  agreements  concerning 
conformance  testing  shall  not: 

1)  Grant  exclusive  rights  to  others  In  fulfilling  its  responsibilities  in  the  areas 
described  above. 

2)  Unilaterally  agree  to  adopt  a product  or  service  which  conflicts  with,  or  that  does 
not  allow  for  changes  to  meet,  the  IGOSS  Panel  requirements. 

9.  THE  QUALITY  OF  THE  IGOSS  TESTING  PROGRAM 

The  Agent  of  the  IGOSS  Panel*  is  committed  to  upgrade/improve  the  IGOSS  Testing 
Program  to  reflect  inputs  provided  by  the  quarterly  IGOSS/GOSIP  Testing  meetings 
and  discussions  with  the  European  Community  and  several  accreditation  bodies  in  the 
world.  The  Agent  of  the  IGOSS  Panel*  with  the  help  of  the  IGOSS  Partners  intends  to 
improve  this  program  to  be  a leader  in  terms  of  quality,  to  provide  useful  information 
the  public  can  trust,  and  to  be  attractive  to  vendors  for  both  marketing  and  production 
quality  improvements. 

The  IGOSS  Testing  program  strives  for  continuous  quality  improvement  in  five  key 
areas:  Abstract  Test  Suites,  Means  of  Testing,  Laboratories,  Product  Registration,  and 
Vendor  Development  Process. 
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9.1  Quality  of  Abstract  Test  Suites 

The  Agent  of  the  IGOSS  Panel*  has  implemented  a system  of  quality  improvement  to 
obtain  completeness  and  suitability  of  the  abstract  test  suites  (ATSs)  and  to  guarantee 
transparency  in  test  campaigns. 

The  Agent  of  the  IGOSS  Panel*  will  collect  all  the  inputs  (defect  reports,  unsuitable  test 
cases)  from  accredited  laboratories  and  from  the  OIW.  The  Agent  of  the  IGOSS  Panel* 
forwards  these  reports  to  the  relevant  ATS  maintenance  authority  as  well  as  to  all  the 
NVLAP  and  SCO  laboratories  to  insure  transparency  in  the  test  campaign  (i.e.,  same 
defects  known  by  all  laboratories). 

9.2  Quality  of  the  Means  of  Testing 

The  Agent  of  the  IGOSS  Panel*  has  implemented  a system  of  quality  improvement  to 
obtain  completeness  and  suitability  of  the  Means  of  Testing  (MOTs)  and  to  guarantee 
transparency  in  test  campalgns(See  Annex  B). 

The  Agent  of  the  IGOSS  Panel*  ensures  that  all  MOTs  are  assessed  by  approved, 
accredited  laboratories.  Common  procedures  are  in  place  to  insure  the  same  level  of 
assessment  and  the  same  format  for  the  "MOT  Qualification  Reports"  based  on  the 
GOSIP  Means  of  Testing  Generic  Test  Plan  [JITC  1].  NIST's  Agent  provides  the  MOT 
qualification  reports  to  the  MOT  suppliers.  The  MOT  suppliers  are  required  to  provide 
the  relevant  MOT  Qualification  Report  to  all  their  NVLAP  and  SCC  laboratory  clients. 
Moreover,  the  Agent  of  the  IGOSS  Panel*  collects  any  new  defect  reports  from  the 
IGOSS  accredited  test  laboratories.  The  Agent  of  the  IGOSS  Panel*  consolidates 
these  defect  reports  and  provides  them  as  feedback  to  the  MOT  suppliers.  MOT 
suppliers  are  required  to  provide  these  new  defect  reports  to  their  NVLAP  and  SCC 
laboratory  clients.  This  system  helps  to  achieve  transparency  in  test  campaigns  and  to 
facilitate  harmonization  of  test  reports. 


9.3  Quality  in  the  Laboratories. 

All  NVLAP  and  SCC  accredited  laboratories  are  required  to  maintain  their  procedures 

fully  compliant  with  ISO  Guide  25  [ISO-114]  and  the  relevant  element  of  ISO  9002 
[ISO-9000].  NVLAP  and  SCC  monitor  the  quality  and  competence  of  the  IGOSS 
Testing  Laboratories. 
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9.4  Quality  of  the  Product  Registration 

The  Agent  of  the  IGOSS  Panel*  is  the  Product  Registration  Authority.  The  Agent  of  the 
IGOSS  Panel*  has  in  place  a comprehensive  and  high-quality  test  report  review 
process.  The  objectives  of  this  review  are  twofold:  1)  the  suitability  of  the  products  to 
the  relevant  profiles  and  2)  the  guarantee  that  the  laboratories  perform  to  the  level  of 
quality  required  (see  Annex  A). 

During  this  process,  the  registration  authority  verifies  that  all  the  documents  provided 
by  the  vendors  follow  the  procedures  defined  in  ISO  9646  [113]  and  comply  to  the 
IGOSS  Testing  program  requirements.  Indeed,  a test  report  must  be  comprehensive, 
complete,  reliable.  Some  of  the  documents  required  Include  the  Protocol 
Implementation  Conformance  Statement  (PICS),  Protocol  Implementation  eXtra 
Information  (PIXIT),  Protocol  Conformance  Test  Report  (PCTR),  System  Conformance 
Test  Report  (SCTR),  Product  Architecture  Diagram,  and  Test  Configuration  Diagram.  A 
block  and  wire  diagram  must  be  provided  that  includes:  1)  a pictorial  explanation  of  the 
product  architecture  which  accurately  depicts  the  product  to  be  registered  and  2)  an 
explanation  of  the  test  configuration  which  accurately  describes  the  software,  hardware, 
and  protocols  used  by  each  components  of  the  product.  Each  test  case  executed  must 
result  in  a behavior  acceptable  from  the  point  of  view  of  the  relevant  International 
Standards  and  Stable  Implementation  Agreements.  When  a test  case  does  not  behave 
as  expected,  the  laboratory  must  demonstrate  that  no  instance  of  non-compliance 
occurs  and  provide  all  necessary  technical  justifications  and  logging  information. 

9.5  Quality  in  the  Vendor  Development  Process 

Vendors  that  have  been  assessed  for  quality  (i.e.,  ISO  9000)  may  be  able  to  enter  their 
status  in  the  IGOSS  Register  Database.  The  plan  would  allow  four  status  levels: 
Unknown,  ISO  9001,  ISO  9002,  and  ISO  9003. 

ISO  9000  (Quality  Management  and  Quality  Assurance  Standards  [ISO  108])  focuses 
on  the  quality  system  In  place  In  the  vendor  organization  In  various  elements  of  the 
product  life-cycle  (I.e.,  design,  development,  production,  final  inspection  and  test. 
Installation,  and  servicing).  The  Introduction  Section  of  ISO  9000  states: 

"A  principal  factor  in  the  performance  of  an  organization  is  the  quality  of 
its  products  or  services.  There  is  a world-wide  trend  towards  more 
stringent  customer  expectations  with  regard  to  quality.  Accompanying 
this  trend  has  been  a growing  realization  that  continual  improvements  in 
quality  are  often  necessary  to  achieve  and  sustain  good  economic 
performance. 
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Most  organizations  - industrial,  commercial  or  governmental  - produce  a 
product  or  service  intended  to  satisfy  a user's  need  and  requirements.  Such 
requirements  are  often  incorporated  in  'specifications'.  However,  technical 
specifications  may  not  in  themselves  guarantee  that  a customer's  requirements 
will  be  consistently  met,  if  there  happen  to  be  any  deficiencies  in  the 
specifications  or  in  the  organizational  system  to  design  and  produce  the  product 
or  service.  Consequently,  this  has  led  to  the  development  of  quality  systems 
standards  and  guidelines  that  complement  relevant  product  or  service 
requirements  given  in  technical  specifications.  The  series  of  International 
Standards  (ISO  9000  to  ISO  9004  inclusive)  embodies  a rationalization  of  the 
many  and  various  national  approaches  in  this  sphere". 

At  the  time  of  registration,  together  with  the  PCTRs  and  SCTRs,  vendors  would  be 
asked  about  the  quality  level  of  the  vendor  organization.  The  Agent  of  the  IGOSS 
Panel*  would  require  the  proof  of  any  claimed  quality  registration.  The  relevant  ISO 
9000  certification  body  would  also  be  referenced.  It  Is  not  the  role  of  the  IGOSS  Testing 
Program  to  assess  the  quality  and  validity  of  these  bodies. 

This  recognition  of  the  quality  in  the  Vendor  Development  Process  Is  not  vet 
implemented. 
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ANNEX  A:  IGOSS  PRODUCT  REGISTRATION 

Note:  In  case  of  conflict,  the  most  updated  and  approved  version  of  the  product 
registration  procedure  take  precedence  on  the  information  provided  in  this  Annex. 
Contact  the  Registration  Authority  for  the  latest  procedure. 

A.1  Registration  Information 

A.1.1  Summary:  Howto  Become  Registered 

A product  cannot  legitimately  be  claimed  as  IGOSS  Compliant  if  it  Is  not  registered. 
The  conditions  for  registration  are  given  below. 

1)  To  be  registered  a product  must  have  been  successfully  conformance  tested  in  a 
NVLAP  accredited  laboratory  (or  a laboratory  which  Is  affiliated  through  a 
Memorandum  of  Understanding)  or  an  SCC  accredited  laboratory.  The  testing 
requirement  applies  to  all  IGOSS  protocols  within  the  product,  with  exceptions  as 
noted  below. 

2)  A partial  stack  product  may  be  registered  if: 

a)  all  IGOSS  protocols  within  it  are  successfully  conformance  tested  (e.g. 
P2,  P1,  RTS,  Session);  and 

b)  the  connectivity  platform(s)  referenced  is  also  registered  (e.g.  TP0/X.25 
PLP/X.25  HDLC  LAP  B). 

3)  The  testing  and  registration  of  LAN  products  (8802/2,  8802/3,  8802/4,  8802/5)  is 
optional:  however,  untested  two-layer  LAN  products  may  not  be  legitimately 
claimed  as  IGOSS  compliant,  whereas  successfully  tested  protocol  stacks 
running  above  untested  two-layer  LAN  products  can  be  legitimately  claimed  as 
IGOSS  compliant.  This  provides  the  exception  to  rule  1),  illustrated  as  follows: 

In  a full  stack  platform  of  P2/P1/RTS/Session/TP4/CLNP/8802.2/8802.3,  CLNP 
through  P2  must  be  tested  and  registered.  The  LAN  connectivity  8802.2/8802.3 
must  be  referenced  in  the  product  application  (as  a specific  product 
Identification)  but  need  not  exist  as  a product  on  the  register  of  tested  LANs. 

4)  There  is  no  testing,  or  registration  requirement  for  physical  layer  protocol 
products. 

5)  Derived  products  may  be  qualified  for  registration  according  to  the  criteria  given 
in  section  A.  1.3  of  this  document. 

Product  suppliers  with  IGOSS  products  to  be  tested  and  registered  should: 

1)  Complete  the  IGOSS  PICS,  or  set  of  PICSs,  for  the  protocols  implemented  in  the 
product. 
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2)  Submit  the  product  for  conformance  testing  to  an  IGOSS  registered  laboratory. 

3)  Ensure  that  all  executed  tests  are  PASSed.  Tests  Not  Run/Selected  are  de- 
selected according  to  the  relevant  criteria. 

4)  Submit  the  PICSs,  PCTRs  and  SCTR  together  with  the  IGOSS  Product 
Registration  application  form  (obtainable  from  Agent  of  the  IGOSS  Panel*  or 
from  the  conformance  laboratory)  to  the  Agent  of  the  IGOSS  Panel*  for  review 
and  registration. 

A.  1.2  Base  Products 

The  definition  of  a IGOSS  "product"  has  been  engineered  to  allow  the  register  to  retain 
a coherent  structure.  Of  course  a supplier  is  at  liberty  to  package  products  in  his  own 
way,  but  this  means  that  It  is  possible  that  what  a supplier  packages  as  one  product 
may  need  to  appear  as  entries  on  multiple  registers.  The  principle  concern  of  the 
IGOSS  Testing  Program  Is  that  protocol  functionality  at  each  layer  be  conformance 
tested,  and  it  is  this  aspect  which  determines  whether  a product  is  a "Base". 

To  clarify:  a Base  Product  is  that  part  of  a supplier's  packaged  software  product  which 
Implements  IGOSS  functionality  and  has  successfully  passed  conformance  testing,  and 
resides  on  a computer  system  with  a named  operating  system,  or  a family  of  computer 
systems  having  the  same  processor  and  instruction  set  and  the  same  operating  system 
as  used  during  the  test  campaign. 

A.1.3  Derived  Product  Guidelines 

(1)  In  certain  circumstances  a IGOSS  Product  may  be  derived  from  a tested  'base' 
implementation,  without  requiring  complete  formal  testing.  In  his  application  for 
derived  product  registration  the  vendor  must  provide  the  rationale  for  his 
assertion  that  the  implementation  Is  substantially  unaltered  (from  the  originally 
tested  base  product).  When  considering  registration  requests  for  derived 
products  the  Agent  of  the  IGOSS  Panel*  will  use  the  following  guidelines. 

(a)  The  host  and  target  computer  systems  of  the  base  and  derived  IGOSS 
Implementations  have  compatible  instruction  sets  and  operating  systems. 
Common  examples  of  compatible  Instruction  sets  and  operating  systems 
are  two  different  computer  system  models  in  a manufacturer's  product  line 
or  the  computer  systems  produced  by  different  manufacturers  that  use 
the  same  hardware  mechanisms  and  operating  systems. 

(b)  The  IGOSS  Implementation  proposed  for  registration  was  derived  from 
the  base  implementation  by  changes  that  are  within  the  scope  of 
accepted  software  maintenance  practices.  Arguments  along  this  line 
should  be  included,  in  writing,  with  the  application. 
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(c)  Basic  Interconnection  Testing  is  conducted  on  the  derived 
implementation,  which  results  in  the  production  of  test  logs  showing  PDU 
exchange  activity  which  is  the  same  as  the  base  implementation  or,  if 
there  are  minor  differences,  these  differences  are  justified  as  being  within 
the  scope  of  accepted  software  maintenance  practices.  Samples  of  the 
test  logs  and  associated  PCTRs  for  the  basic  interconnection  tests  should 
support  the  derived  product  application. 

(d)  The  base  implementation  was  assessed  in  accordance  with  the  Agent  of 
the  IGOSS  Panel*  procedures  and  is  registered  with  the  Agent  of  the 
IGOSS  Panel*. 

(2)  An  IGOSS  Product  may  be  registered  for  use  over  a different  communications 
platform  than  that  originally  registered,  without  complete  formal  testing.  When 
considering  registration  requests  the  Agent  of  the  IGOSS  Panel*  will  use  the 
following  guidelines. 

(a)  The  base  implementation  was  assessed  in  accordance  with  the  Agent  of 
the  IGOSS  Panel*  procedures  and  is  registered  with  the  Agent  of  the 
IGOSS  Panel*. 

(b)  The  communications  platform  to  which  the  implementation  is  ported  was 
assessed  In  accordance  with  the  Agent  of  the  IGOSS  Panel*  procedures 
and  Is  registered  with  the  Agent  of  the  IGOSS  Panel*. 

(c)  Basic  Interconnection  Testing  is  conducted  on  the  ported  platform  which 
results  In  the  production  of  test  logs  showing  PDU  exchange  activity  which 
is  the  same  as  the  base  Implementation  or.  If  there  are  minor  differences, 
these  differences  are  justified  as  being  within  the  scope  of  accepted 
software  maintenance  practices.  Samples  of  the  test  logs  should  support 
the  ported  platform  application. 

A derived  or  ported  implementation  may  lose  its  registration  if  It  Is  challenged 
successfully.  Such  challenges  are  described  in  Section  A.2. 

A.  1.4  Derived  Product  Guideline  Analysis 

One  of  the  most  significant  activities  in  the  IGOSS  program  Is  that  of  placing  products 
on  the  IGOSS  register.  The  IGOSS  program  provides  two  types  of  registration;  base 
and  derived.  The  "base”  registration  Is  the  initial  registration  for  a product  and  is 
achieved  only  after  the  product  has  been  fully  tested  by  an  accredited  lab.  The 
"derived"  registration  can  be  obtained  when  a vendor  can  definitively  prove  that  a 
subsequent  version  of  that  product  is  essentially  unchanged  from  its  base  version.  This 
is  done  without  the  product  having  to  undergo  the  same  levels  of  testing  as  for  a base 
registration. 
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The  purpose  of  this  section  is  to  provide  both  general  and,  where  possible,  specific 
guidelines  for  derived  registrations.  In  the  attempt  to  quantify  the  conditions  for  derived 
registrations  it  was  determined  that  the  problem  is  too  complex  to  allow  for  any 
reasonably  simple  guidelines.  The  greater  success  has  been  in  guidelines  for  specific 
situations  which  comprise  only  a subset  of  the  possibilities.  As  vendors  identify  new 
situations  each  will  be  examined  individually  and  ruled  upon  by  the  registration 
authority.  Input,  suggestions,  and  disagreements  relative  to  this  topic  Is  requested. 

It  is  the  position  of  the  Agent  of  the  IGOSS  Panel*  that  in  order  to  register  an 
Implementation  as  a derived  product  it  must  be  demonstrated  that  there  are  no 
significant  differences  between  it  and  the  base  product  from  which  it  is  being  derived. 
The  process  of  determining  this  deals  with  three  significant  entities  called  the  protocol 
engine,  the  virtual  machine,  and  the  underlying  stack  (i.e.,  connectivity). 

The  protocol  engine  Is  that  portion  of  the  product  that  Is  defined  by  the  PICS  and  tested 
by  an  accredited  test  lab  for  IGOSS  conformance.  It  is  the  heart  of  the  registered 
product  and  cannot  be  modified  in  any  way  without  requiring  a complete  re-test  for  a 
subsequent  IGOSS  product  registration.  Therefore,  the  initial  condition  for  a derived 
registration  is  that  the  PICS  for  the  new  product  version  is  identical  to  the  PICS  for  the 
base  registered  version.  Any  change  In  the  PICS  Implies  a change  to  the  protocol 
engine  which  requires  complete  conformance  testing. ' 

The  virtual  machine  is  the  environment  in  which  the  protocol  engine  operates  and 
consists  of  the  hardware  platform  and  the  operating  system.  Changes  to  the  virtual 
machine  which  affect  the  protocol  engine  (i.e.,  not  transparent)  will  not  be  allowed 
without  a complete  re-test  of  the  candidate  product.  All  changes  to  the  virtual  machine 
will  have  to  be  carefully  reviewed  before  a derived  registration  can  be  allowed.  It  is 
incumbent  upon  the  product  vendor  to  provide  complete  justification  of  any  claims  for 
derived  registration  when  the  virtual  machine  is  affected. 

The  underlying  stack,  or  connectivity,  is  that  set  of  software,  firmware,  and  hardware 
that  provide  the  services  required  by  the  protocol  engine  to  connect  through  the  LAN  or 
WAN  to  its  peer(s).  For  example,  TP4  can  connect  through  CLNP  over  802.2  (LLC) 
over  802.3  (MAC).  The  two  changes  that  occur  relative  to  the  substack  are  connectivity 
or  implementation  (I.e.,  same  protocol,  but  different  product)  changes.  All  changes  to 
the  underlying  stack  (connectivity)  will  also  have  to  carefully  reviewed  before  a derived 
registration  can  be  allowed. 

In  order  for  the  Registration  Authority  to  grant  a derived  registration  for  a product  that 
meets  the  requirement  that  the  change(s)  be  transparent  to  the  protocol  engine,  the 
vendor  must  provide  both  conclusive  proof  of  that  the  change  was  transparent  and  a 
demonstration  of  testing  at  the  "derived"  level.  This  "derived"  test  suite  will  be  a subset 
of  the  ATS  which  is  sufficient  to  provide  confidence  that  the  claim  for  a derived 
registration  is  valid.  The  results  of  this  test  must  be  provided  to  the  Registration 
Authority  In  the  same  PCTR  format  used  for  base  registrations. 
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Where  an  ATS  or  MOT  does  not  exist  (e.g.,  802.4,  802.5)  with  which  to  provide  this 
"derived"  test  it  will  be  necessary  for  the  vendor  to  perform  an  "interoperability"  test 
between  the  base  and  derived  product.  This  test  should  be  conducted  using  the 
appropriate  "mandatory"  OSINET  tests  for  two  high  level  protocols,  such  as  MHS  or 
FTAM.  A description  of  the  test  and  results  must  be  provided  in  the  place  of  the 
"derived"  test  results. 

Unless  otherwise  specified  in  a policy  statement,  no  product  will  be  considered  for  a 
derived  registration  without  the  test  report  from  a successful  "derived"  or 
interoperability  test.  Irrefutable  written  evidence  that  the  protocol  engine  has  not  been 
affected  by  whatever  modification  Is  being  considered,  and  a PICS  that  is  the  same  as 
the  one  for  the  base  product.  The  following  are  examples  of  circumstances  where 
derived  registrations  may  or  may  not  be  considered  given  that  all  of  the  previously 
defined  conditions  have  been  met. 

Registrations  Relative  to  the  Virtual  Machine:  Changing  the  virtual  machine,  which  Is 
made  up  of  the  hardware  and  operating  system,  requires  complete  conformance 
testing.  Usually,  changing  the  operating  system  (e.g.,  DOS  to  UNIX),  system  calls,  or 
the  platform  affect  the  protocol  engine  and  are  generally  excluded  from  derived 
registrations.  However,  some  cases  have  been  identified  in  which  hardware  or 
operating  system  changes  might  occur  without  changing  the  virtual  machine  as  viewed 
by  the  protocol  engine. 

Derived  registrations  will  be  considered  for  platform  changes  If  there  is  conclusive 
evidence  presented  by  the  applicant  that  the  protocol  engine  Is  unchanged  because  It 
is  isolated  by  the  operating  system  from  the  platform  architecture  (e.g.,  the  POSIX 
operating  system),  but  care  should  be  given  to  consider  timing  changes. 

Example: 

product/POSIX/systemX  ==>  product/POSIX/systemY 

Derived  registrations  will  be  considered,  without  testing,  if  there  is  conclusive  evidence 
presented  by  the  applicant  that  there  is  no  difference  In  the  hardware  architecture.  For 
example,  the  difference  In  model  numbers  is  due  to  different  peripherals  being 
configured  with  the  same  processor,  chassis,  etc. 

Example: 

product/OpSys/pIatformX.1  ==>  product/OpSys/platformX.2 

Derived  registrations  will  be  considered  if  there  Is  conclusive  evidence  presented  by  the 
applicant  that  there  were  minor  modifications  In  the  operating  system  (e.g.,  minor 
version  changes)  which  were  transparent  to  the  protocol  engine. 

Example: 

prod uct/OpSysX.1 /platform  ==>  product/OpSysX.2/p!atform 
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Registrations  Relative  to  Interface  Changes:  Derived  registrations  will  be  considered 
where  the  substack  is  changed  from  one  registered  substack  product  to  another  when 
the  protocol  connectivity  remains  unchanged.  In  other  words,  given  that  two,  or  more, 
identical  registered  products  are  available  for  a given  protocol  it  is  allowed  to  change 
from  one  to  another  and  to  derive  the  registration  of  the  second  stack  since  the 
connectivity  remains  unchanged. 

Example: 

TP4/CLNP(brand  x)/LAN  ==>  TP4/CLNP(brand  y)/LAN 

CLNP  Registration  Relative  to  MAC  Sub  layer  Changes:  Because  the  CLNP  protocol  Is 
Insulated  from  MAC  changes  it  is  possible  to  "derive"  register  a conformant  CLNP 
product  from  the  MAC  (802.3,  802.4,  802.5)  It  was  tested  over  to  another  MAC  given 
that  the  LLC  sub  layer  remains  unchanged. 

Example: 

CLNP/802.2/802.3  ==>  CLNP/802.2/802.4 

CLNP  Registration  Relative  to  LANA/VAN  Changes:  Because  the  SNDCF  (Sub  Network 
Dependent  Convergence  Function)  is  tested  with  the  CLNP  It  should  be  considered  part 
of  the  CLNP  which  excludes  derived  registrations  from  LAN  to  WAN  or  visa  versa.  The 
purpose  of  the  SNDCF  is  to  create  a generic  interface  between  CLNP  and  the  Service 
Provider  (LLC1  or  X.25).  Because  the  SNDCF  differs  between  LAN/WAN,  the 
connectivity  change  is  not  transparent  to  the  CLNP  protocol  engine. 

An  investigation  of  CLNP  shows  that  when  It  Is  implemented  over  LLC  that  the  LLC 
assumes  the  bulk  of  the  SNDCF  functionality.  In  this  situation  the  SNDCF  is  a simple 
interface  which  passes  the  CLNP  requests  and  LLC  responses  back  and  forth  between 
the  CLNP  and  the  LLC  with  whatever  format  changes  are  required.  When  CLNP  Is 
implemented  over  X.25  the  SNDCF  provides  a significant  amount  of  functionality  by 
both  translating  the  requests  and  providing  the  X.25  connection  control.  It  is  obvious 
that  there  is  no  similarity  between  the  X.25  SNDCF  and  the  LAN  SNDCF  and  hence 
CLNP/LLC  and  CLNP/X.25.  Since  no  mechanism  exists  to  isolate  the  SNDCF  for 
testing  it  Is  necessary  to  require  a Base  registration  for  this  connectivity  change. 

TP4  Registration  Relative  to  CLNP/WAN  CLNP/LAN  Changes:  Because  the  TP4 
protocol  Is  Insulated  from  the  WAN/LAN  changes  it  is  possible  to  derive  register  a 
conformant  TP4  product  from  the  LAN  or  WAN  It  was  tested  over  to  the  other  given  that 
the  CLNP  sub  layer  was  tested  and  registered  over  both. 

There  are  two  conditions  under  which  this  might  happen.  One  is  that  the  CLNP  was 
only  registered  over  one  of  the  substacks  when  the  TP4  was  tested  and  registered  over 
It.  The  other  Is  that  the  CLNP  was  registered  over  both  substacks,  but  the  TP4  was 
only  tested  and  registered  over  one.  In  the  first  case,  the  TP4  could  be  derived  to  the 
additional  substack  after  the  CLNP  was  "re-registered"  over  both  substacks.  In  the 
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second  case,  the  TP4  could  be  tested  and  registered  over  one  substack  and  derived  to 
the  other. 

Example  A: 

(1)  base  registration  TP4/CLNP/LAN  (CLNP  only  over  LAN) 

(2)  CLNP  is  tested  and  registered  for  both  LAN  and  WAN 

(3)  ==>  TP4/CLNP/WAN 

Example  B: 

(1)  base  registration  TP4/CLNP/LAN  (CLNP  over  LAN  & WAN) 

(2)  ==>  TP4/CLNP/WAN 

TP4  Registration  Relative  to  CONS  (X.25)  and  CLNS  (CLNP):  Because  there  is  a 
difference  in  protocol  behaviors  between  TP4/CONS  (TP4/X.25)  and  TP4/CLNS 
(TP4/CLNP)  it  is  not  allowed  to  do  a derived  registration  between  them.  TP4/CONS  and 
TP4/CLNS  are  generally  separate  implementations  of  the  protocol  engine. 

Registrations  Relative  to  TP4  to  TP2/TP0  Changes:  Because  there  is  a difference  in 
protocol  behaviors  between  TPO,  TP2,  and  TP4  it  is  not  allowed  to  do  a derived 
registration  between  TP4,  TP2,  or  TPO. 

FTAM  Registrations  Relative  to  TP4,TP2  and  TPO:  Because  the  Service  Provider 
Interface  provided  by  the  transport  layer  appears  to  be  the  same  for  TPO,  TP2  and  TP4 
it  Is  possible  to  derive  register  an  FTAM/TPO  product  from  either  an  FTAM/TP2  or 
FTAM/TP4  product  or  derive  an  FTAM/TP2  product  from  an  FTAM/TP4  given  that  the 
Transport  product  are  both  IGOSS  registered. 

Example: 

FTAM/TP4/CLNP  ==>  FTAM/TP0/X.25 

Product  Registrations  Relative  to  User  Interface  Changes:  Because  the  protocol  engine 
Is  insulated  from  most  User  Interface  changes  it  is  possible  to  derive  register  a product 
from  a conformant  base  given  that  the  protocol  engine  Is  unchanged. 

Product  Registrations  Relative  to  Version  Changes:  Because  product  version  changes 
are  often  due  to  modifications  transparent  to  the  protocol  engine  it  Is  possible  to  derive 
register  a product  from  a conformant  base  given  that  the  protocol  engine  is  unchanged. 

A.  1.5  Mutual  Recognition  Arrangements 

An  IGOSS  product  registered  or  certified  by  another  authority  may  be  registered  with 
the  Agent  of  the  IGOSS  Panel*  if: 

(a)  A Memorandum  of  Understanding  exists  between  the  Agent  of  the  IGOSS 
Panel*  and  the  registering  or  certification  authority,  recognizing  each  other's 
assessment  and  registration  procedures.  Among  other  things  this  MOU  Is 
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dependent  on  mutual  recognition  arrangements  between  NV1_AP  and 
corresponding  accreditation  bodies. 

(b)  Specific  application  is  made  to  the  Agent  of  the  IGOSS  Panel*  by  the  IGOSS 
product  supplier.  (Automatic  concatenation  of  registers  will  not  occur). 

(c)  The  candidate  IGOSS  product  for  registration  passes  the  Agent  of  the  IGOSS 
Panel*  static  evaluation. 

(d)  The  candidate  IGOSS  product  has  not  been  removed  from  the  Agent  of  the 
IGOSS  Panel*  register  as  a result  of  a successful  challenge  which  has  not  been 
resolved  by  the  primary  registration  or  certification  authority. 

If  the  above  conditions  are  met,  the  candidate  IGOSS  product  will  be  registered.  If  a 
IGOSS  product  which  is  registered  under  the  provisions  of  a Mutual  Recognition 
Arrangement  is  removed  from  the  primary  authority's  register  then  it  shall  be  removed 
from  the  IGOSS  register. 

The  Agent  of  the  IGOSS  Panel*  will  not  require  full  testing  of  Implementations  assessed 
by  a mutually  recognized  authority.  The  Agent  of  the  IGOSS  Panel*  will  review  the 
application  for  indications  of  satisfaction  of  IGOSS  requirements.  The  IGOSS  Panel  Is 
the  final  authority  on  whether,  and  In  what  circumstances,  registration  shall  occur. 

Applications  which  are  acceptable  to  the  Agent  of  the  IGOSS  Panel*  are  entered  in  the 
register  of  IGOSS  conformance  tested  products  under  the  same  categorization  as  the 
primary  registration  (i.e..  Registered  or  Provisionally  Registered)  and  with  the  same 
constraints  - Registration  is  based  on  the  given  Registered  Abstract  Test  Suite,  The 
IGOSS  product  supplier  will  be  notified  when  registration  is  approved.  The  register  will 
be  maintained  up-to-date  by  the  Agent  of  the  IGOSS  Panel*  and  the  Information  will  be 
available  according  to  the  procedures  given  in  Annex  C. 

A.1.6  Product  Register  Structure 

Most  OSI  products  are  multi-layered  so  each  separately  identified  product,  as  defined 

by  the  supplier,  is  registered  according  to  the  highest  layered  protocol  which  it  contains. 
For  every  protocol  for  which  IGOSS  compliance  Is  claimed,  a PCTR  must  be  provided. 
This  must  either  be  submitted  as  part  of  the  product  for  which  registration  is  sought,  or 
referenced  as  part  of  a registered  platform.  For  Local  Area  Network  interfaces 
underlying  CLNP,  a non-IGOSS  supporting  platform  will  be  considered  on  a case-by- 
case basis.  These  circumstances  will  only  occur  In  certain  platforms  supporting  CLNP. 

A product  may  be  composed  of  one  or  more  Testable  Units.  Testable  Units  are  defined 
as:  Product  = "exposed"  Testable  Unit 

+ (zero  or  more)  "embedded"  Testable  Unit(s) 

+ Supporting/  Registered  Platform 
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Register  categories  are  numbered  P-1  through  P-n.  The  following  list  is  provided  for 
information  only  and  reflects  the  numbering  for  GOSIP  Version  2. 

P-1  WAN  Products: 

X.25  PLP  + HDLC  LAP  B 
CONS/X.25  PLP  + HDLC  LAP  B 
ISDN 

P-2  LAN  Products: 

8802.2  LLC  + 8802.3  MAC 

8802.2  LLC  + 8802.4  MAC 

8802.2  LLC  + 8802.5  MAC 

P-3  Intermediate  Systems: 

CLNP  + Supporting  LANAA/AN  Platforms 
CLNP+ES-IS  + Supporting  LAN/WAN  Platforms 

P-4  Transport  Products: 

TP4  + CLNP  + Supporting  LAN  Platform(s) 

TP4  + CLTP  + CLNP  + Supporting  LAN  Platform(s) 

TP4  + CLNP  + Registered  WAN  Platform (s) 

TP4  + Registered  WAN  Platform 
TPO  + Registered  WAN  Platform 

P-5  Session  Products: 

Exposed  Session  + Registered  Transport  Platform(s) 

P-6  X.400  Products: 

PI  + RTS/Session  + Registered  Transport  Platform 
PI  + RTS  + Registered  Session  Platform 
P2  + PI  + RTS/Session  + Registered  Transport  Platform 
P2  + PI  + RTS  + Registered  Session  Platform 
P2  + Registered  PI  Platform 

P-7  FTAM  Products: 

FTAM/ACSE/Presentation/Session  + Reg.  Transport  Platform 
FTAM/ACSE/Presentation  + Registered  Session  Platform 

P-8  Virtual  Terminal  Products: 

VTP/ACSE/Presentation/Session  + Reg.  Transport  Platform 
VTPM/ACS E/Presentation  + Registered  Session  Platform 

Register  fields  are: 

Supplier  Name: 
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Contact  Name: 

GOSIP/IGOSS  Product  Name,  Release  and  Date: 

Hardware  and  Operating  System  Platform (s): 

Base/Derived: 

Protocols  and  Profiles: 

Supporting  GOSIP  Platform: 

Date  Registered: 

ATSs  Used: 

MOTS  Used: 

GOSIP/IGOSS  Version: 

Type  of  Registration: 

Type  of  Testing  Laboratory  (1st,  2nd  or  3rd  Party): 

Observations/Restrictions: 

A.  1.7  Registration  Categories 

Products  must  be  successfully  conformance  tested  in  a NVLAP  or  SCC  Accredited 
laboratory  (or  a laboratory  which  is  affiliated  through  a Memorandum  of 
Understanding),  using  registered  Means  of  Testing.  The  number  of  products  registered 
is  not  limited.  Categories  are: 

Registered  - Valid  for  the  life  of  the  product  or  until  major  revision.  Based  on  fully 
registered  Means  of  Testing  and  fully  registered  Abstract  Test  Suites. 

Provisionally  Registered  - Valid  for  the  life  of  the  product  or  until  major  revision,  or 
upgrade  to  Full  Registration  by  testing  against  a Fully  Registered  MOT.  Based  on 
provisionally  registered  Means  of  Testing  and/or  provisionally  registered  Abstract  Test 
Suites. 

Discussion 


The  difference  between  Fully  Registered  and  Provisionally  Registered  products  Is  In  the 
quality  of  testing  undergone.  Initially,  products  are  tested  against  Provisional  MOTs 
and  thus  are  Provisionally  Registered  themselves.  When  the  MOTs  are  upgraded  to 
Full  Registration,  Products  subsequently  tested  implicitly  provide  a higher  assurance  of 
IGOSS  compliance.  It  is  at  the  vendor's  discretion  whether  and  when  a product  shall 
be  upgraded  by  re-testing.  Provisional  and  Fully  Registered  products  coexist  on  the 
register,  but  new  procurements  are  advised  to  favor  Fully  Registered  products, 
wherever  possible. 

The  Agent  of  the  IGOSS  Panel*  Intends  to  make  registers  of  IGOSS  conformant 
products  available  to  the  public  through  quarterly  publication  and  on-line  access.  If  the 
supplier  objects  to  such  procedures,  a written  statement  should  be  Included  with  each 
application. 

A.1.8  IGOSS  Implementation  Philosophy 
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In  establishing  the  registration  scheme  for  the  IGOSS  testing  program,  the  Agent  of  the 
IGOSS  Panel*  was  compelled  to  structure  the  conformance  tested  products  register 
according  to  the  'testable  units'  of  the  7-layer  OSI  stack.  This  structure  may  seem  to 
constrain  the  Implementation  choices  of  a IGOSS  product  supplier,  but  that  is  not  the 
intention  of  the  IGOSS  Panel,  and  not  at  all  the  spirit  of  OSI.  The  nature  of  distributed 
implementation  implies  the  possibility  that  what  OSI  defines  as  an  "End  System"  may 
be  implemented  over  multiple  processors.  The  nature  of  the  interprocess 
communication  between  layers  within  an  OSI  stack,  and  between  an  OSI  stack  and  its 
user  interface,  may  be  tightly  or  loosely  coupled,  and  Is  not  constrained  by  the  IGOSS 
specification.  Examples  of  acceptable  OSI  Implementation  strategies  are: 

1)  A multi-processor  system  Implements  transport  on  a "front-end"  and  applications 
(FTAM->Session,  for  instance)  on  separate  systems,  communicating  over  a bus 
or  a l_AN. 

2)  A network  file  server  implements  FTAM  and  supporting  OSI  protocols  on  one 
system,  with  user  access  by  multiple  workstations  using  any  local  means  of 
interprocess  communication  including  communicating  over  a LAN. 

3)  In  a messaging  system,  the  Message  Transfer  Agent  is  Implemented  on  one 
system  and  User  Agents  are  deployed  in  separate  workstations.  The  MTAs  and 
UAs  communicate  over  a LAN. 

A.2  Appeals 

If  an  Acquisition  Authority  finds  a defect  In  a registered  IGOSS  product,  then  the 
primary  recourse  must  be  to  the  product  supplier.  If  the  Acquisition  Authority  remains 
dissatisfied,  then  details  may  be  given  to  the  Agent  of  the  IGOSS  Panel*,  and  the 
product  supplier  may  be  required  to  demonstrate  conformance  with  respect  to  a given 
test  or  tests.  If  the  supplier  is  unable  to  demonstrate  conformance  In  those  cases,  then 
the  product's  registration  may  be  withdrawn.  For  each  test  or  set  of  tests  appealed,  the 
appellant  is  required  to  pay  a nominal  fee. 

In  the  case  of  a derived  product,  if  the  supplier  is  unable  to  demonstrate  conformance 
with  respect  to  any  tests,  then  the  derived  registration  may  be  revoked.  In  such  a case, 
a derived  registration  cannot  be  reinstated:  the  product  must  undergo  a complete 
conformance  test,  to  achieve  registration  as  a base  Implementation. 

A.3  Related  Conformance  Testing  Information 

Prior  to  entry  onto  the  register  of  IGOSS  Conformance  Tested  Products  it  is  necessary 
for  a product  to  be  successfully  conformance  tested  In  a NVLAP  or  SCC  accredited 
laboratory,  using  an  IGOSS  registered  Means  of  Testing.  IGOSS  Testing  Program 
policy  has  been  to  promote  improvements  in  the  underlying  testing  infrastructure;  in 
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practice  this  involves  migration  of  Abstract  Test  Suites,  and  updating  of  Means  of 
Testing,  with  the  result  that  earlier  ATS  and  MOTs  become  obsolete.  It  is  important  for 
a product  supplier  to  know  the  ATS  and  MOT  obsolescence  policy  before  attempting 
registration  - indeed,  before  initiating  formal  testing. 

Related  to  the  testing  Improvement  Issue  Is  the  treatment  of  defects  In  Abstract  and 
Executable  test  suites.  Registration  can  occur  only  after  the  appropriate  tests  have 
been  selected  and  successfully  executed.  It  is  important  for  both  the  test  laboratory 
and  the  Registration  Authority  to  have  the  same  picture  of  Abstract  and  Executable 
defects.  ISO  9646  assumes  perfection  in  all  test  cases  - and  the  IGOSS  Testing 
Program  is  painfully  aware  that  the  situation  is  otherwise.  Hence,  additional  procedures 
for  the  de-selection  of  test  cases  are  given  below. 

A.3.1  Test  Case  De-selection  Procedures 

Test  de-selection  may  be  based  on:  1)  non-implementation  of  optional  capabilities,  2) 
Inapplicability  of  tests  for  IGOSS.  Tests  selected  but  not  run  may  be  based  on:  3) 
Abstract  Test  Case  defects,  4)  Ambiguities  and  Errors  In  the  Standards  leading  to 
Abstract  Test  Case  defects,  5)  Executable  Test  Case  defects  (generally  applicable,  and 
acknowledged  by  the  MOT  supplier),  and  possibly  6)  Executable  Test  Case  defects 
(specific  to  an  lUT,  and  maybe  acknowledged  by  the  MOT  supplier). 

A review  of  the  application  of  the  test  case  de-selection  mechanisms  is  appropriate: 

1)  Non  Implementation  of  Optional  Capabilities 

De-select. 

Optional  capabilities  not  claimed  need  not  be  tested,  of  course.  Note 
however  that  some  options  may  be  IGOSS  Requirements.  Non 
Implementation  of  these  optional  capabilities  will  result  In  failure  of  Static 
Conformance. 

Optional  capabilities  implemented  must  be  tested,  if  a test  case  exists, 
except  for  de-selection  based  on  points  2)  to  6). 

2)  Inapplicability  of  Test  for  IGOSS 

De-select. 

This  situation  exists  In  cases  where  the  test  suite  was  developed  In 
Europe  based  on  a European  profile,  and  IGOSS  points  to  a version  of 
the  OIW  Implementors  Agreements  which  is  not  aligned.  A review  of  the 
Abstract  Test  Cases  In  1990  was  supposed  to  highlight  these  kinds  of 
problems,  but  that  review  was  not  sufficiently  effective. 
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Any  such  tests  identified  today  may  be  de-selected,  with  written  justification  of 
non-applicability.  The  registration  authority  will  generate  a list  of  all  such  tests. 
These  tests  are  excluded  from  what  constitutes  "100%"  in  determining  whether 
an  MOT  supports  the  required  threshold  percentage  of  coverage. 

3)  Abstract  Test  Case  Defects 

(a)  Serious  Abstract  Test  Case  Defects 

Selected  and  Not  Run. 

Any  such  tests  identified  today  need  not  be  run,  based  on  written 
justification  of  an  ATC  defect  from  the  registration  authority.  The 
registration  authority  maintains  a list  of  all  such  tests. 

These  are  tests  which  are  "temporarily  excluded"  from  the  requirement  for 
execution,  pending  the  application  and  promulgation  of  a "fix"  by  the 
appropriate  Test  Suite  maintenance  authority. 

(b)  Trivial  Abstract  Test  Case  Defects 

Selected  and  Run 

Any  Abstract  Test  Cases  which  are  labeled  as  defective,  but  for  which  the 
defect  is  superficial,  or  for  which  a valid  workaround  exists,  must  be  run. 
If  necessary,  verdicts  may  be  assigned  after  manual  analysis.  The 
registration  authority  maintains  a list  of  all  such  test  cases. 

4)  Ambiguities  and  Errors  In  the  Standards  leading  to  Abstract  Test  Case 
Defects 

Selected  and  Not  Run. 

Any  such  tests  identified  today  need  not  be  run,  based  on  written 
justification  of  non-applicability.  This  justification  should  include  an 
interoperability  impact  statement,  since  it  is  likely  that  such  specification 
problems  will  be  variously  treated  by  different  product  implementors.  It 
may  be  useful  to  distinguish  between  "positive"  testing  and  "negative" 
testing  here,  due  to  their  possibly  different  interoperability  consequences: 

The  purpose  of  positive  testing  is  to  determine  that  an  lUT  can  initiate 
good  protocol,  and  respond  correctly  to  good  protocol.  The  problems  of 
ambiguous  specification,  when  not  tested,  or  incorrectly  tested,  can  result 
in  serious  interoperability  consequences.  Testing  laboratories  and  IGOSS 
test  report  reviewers  are  advised  to  carefully  consider  these 
consequences  prior  to  registration. 
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The  purpose  of  negative  testing  is  to  determine  that  an  lUT  can  respond 
correctly/in  a well-mannered  fashion  to  errors  initiated  by  the  MOT.  This 
usually  results  in  either  ignoring  the  offending  PDU,  or  initiating  an  abort 
sequence.  In  general,  the  problems  of  ambiguous  specification,  when 
exercised  by  negative  testing,  are  more  likely  to  be  self-resolving  by 
resulting  In  an  abort. 

The  registration  authority  is  advised  to  generate  a list  of  all  such 
ambiguous  specification  areas  and  associated  tests  for  submission  to  the 
relevant  standards  making  authorities,  including  the  OSE  Implementors' 
Workshop. 

These  are  tests  which  are  "temporarily  excluded"  from  the  requirement  for 
execution,  pending  the  application  and  promulgation  of  a "fix"  by  the 
appropriate  Test  Suite  maintenance  authority.  These  tests  are  included 
in  what  constitutes  "100%"  in  determining  whether  an  MOT  supports  the 
required  threshold  percentage  of  coverage. 

5)  Executable  Test  Case  Defects 

(a)  Serious  Executable  Test  Case  defects  (generally  applicable,  and 
acknowledged  by  the  MOT  Supplier  and  the  Registration  Authority) 

Selected  and  Not  Run. 

Any  such  tests  need  not  be  run,  subject  to  written  justification  and 
corroboration  by  the  MOT  Supplier.  This  is  the  purpose  of  having  the 
MOT  Supplier  issue  synchronized  bug  lists  to  their  users  and  to  the 
Registration  Authority. 

It  is  the  responsibility  of  the  MOT  Supplier  to  fix  these  tests. 

(b)  Trivial  Executable  Test  Case  defects  (both  general  and  specific) 

Selected  and  Run 

Any  Executable  Test  Cases  which  are  labeled  as  defective,  but  for  which 
the  defect  is  superficial,  or  for  which  a valid  workaround  exists,  must  be 
run.  If  necessary,  verdicts  may  be  assigned  after  manual  analysis.  The 
registration  authority  coordinates  lists  of  all  such  test  cases,  for  every 
MOT. 

It  Is  the  responsibility  of  the  MOT  Supplier  to  fix  these  tests. 
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6)  Serious  Executable  Test  Case  defects  (specific  to  an  lUT,  and  acknowledged 
by  the  MOT  Supplier  and  the  Registration  Authority) 

Selected  and  Not  Run. 

This  is  possibly  the  thorniest  category  of  all,  and  most  subject  to  potential 
abuse.  Many  of  the  errors  in  this  category  may  be  alternatively  dealt  with 
under  the  "Re-Assignment  of  Verdict"  rubric.  Any  such  errors  which  are 
not  acknowledged  by  the  MOT  Supplier  may  be  reviewed  on  a case  by 
case  basis,  and  accepted  or  refused  by  the  Registration  Authority,  based 
on  the  best  available  technical  opinion.  A different  weight  may  be  placed 
on  the  claims  depending  on  whether  It  Is  a "positive"  or  a "negative"  test 
which  Is  sought  to  be  withdrawn.  Any  such  errors  may  also  be  brought 
before  the  01 W CT  and  relevant  protocol  SIGs,  whose  decision  will  be 
considered  by  the  IGOSS  Testing  Program. 

will  be  accepted  by  the  IGOSS  Testing  Program.  It  may  be  the 
responsibility  of  the  MOT  Supplier  to  fix  these  tests. 

A.3.2  The  Distinction  Between  "Serious"  and  "Trivial"  Errors 

Any  Abstract  or  Executable  Test  Case  defect  detected  by  the  Conformance  Laboratory 
must  be  either  listed  with  the  MOT  Supplier  and  the  Registration  Authority  a priori,  or 
else  a Discrepancy  Report  (DR)  submitted  to  the  Registration  Authority  upon  the 
discovery  of  the  error  (and  its  seriousness),  prior  to  the  submission  of  the  test  report  for 
registration.  The  Registration  Authority  will  rule  on  the  validity  of  the  DR  according  to 
the  best  technical  opinion  available,  including  If  necessary,  a review  under  the  terms  of 
the  OIW  CT  SIG  dispute  resolution  process. 

Any  test  listed  as  defective  or  discovered  to  be  defective  must  be  attempted  to  run  and 
a manual  verdict  assessed.  If  the  test  purpose  is  not  achievable,  then  the  error  is 
"Serious"  and  the  test  may  be  marked  as  "Selected  and  Not  Run".  If  the  test  purpose  is 
achievable,  then  the  relevant  verdict  may  be  recorded  In  the  PCTR. 

A.3.3  Reassignment  of  Verdicts 

IS  9646  indicates  that  Pass  or  Fail  verdicts  in  an  abstract  test  case  may  not  be  varied 
by  the  test  laboratory.  This  is  obviously  an  ideal  situation  where  mature,  standardized 
test  suites  are  in  use,  which  is  not  the  case  today,  further,  it  does  not  take  Into  account 
imperfect  mappings  between  abstract  and  executable  test  cases.  The  situation  today  Is 
that  many  of  the  abstract  test  suites  have  serious  defects  which  require  critical  review 
with  respect  to  the  protocol  standards  and  Implementors  Agreements.  In  these  cases, 
ongoing  review  by  the  test  laboratories  Is  necessary  to  improve  the  test  suite 
standards.  Many  of  the  executable  test  cases  have  defects,  some  of  which  are 
discovered  by  the  MOT  Validation  Laboratory. 
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In  the  case  of  executable  defects,  the  verdict  should  be  changed  to  comply  with  the 
associated  abstract  test  case. 

In  the  case  of  abstract  test  case  defects,  the  verdict  should  be  determined  in 
accordance  with  the  Stable  Implementors'  Agreements  and/or  the  International 
Standard.  The  registration  authority  will  periodically  produce  addenda  to  the  ATS  to 
clarify  problems  discovered  therein. 

A.4  Product  Registration  Checklist 

(1)  Reliable  Test  Report(s) 

The  Test  Report,  as  a record  of  the  products  behavior,  must  be  reliable  to  the 
point  that  a potential  Customer  can  determine  the  product's  capabilities.  In  order 
to  do  this  the  report  must  be  readable,  describe  what  was  tested  versus  what 
capabilities  were  claimed,  and  precisely  identify  the  substack  over  which  It  was 
tested. 

(2)  Reproducibility  of  Tests 

The  test  reports  must  provide  adequate  information  about  the  test  configuration 
and  conduct  to  allow  for  the  reconstruction  and  execution  of  the  test  with 
predictable  results.  In  order  to  do  this  the  test  configuration  must  be  specified  in 
the  PCTR,  SCTR,  PICS,  and  PIXIT.  This  includes  a detailed  specification  of  the 
hardware,  software,  and  substack  connectivity  (i.e.,  protocol(s),  software, 
communication  boards).  (The  SCTR,  PCTR,  and  PICS  may  reference  only  the 
hardware/software  configuration  that  the  product  was  tested  on.) 

When  a single  "base"  application  Is  for  multiple  hardware  platforms,  operating 
systems,  etc.  it  is  essential  that  each  virtual  machine  be  equivalent  to  the  tested 
system.  These  additional  platforms  should  only  be  referenced  In  the  application 
with  information  appended  to  the  application  that  absolutely  proves  that  the 
equivalencies.  They  should  not  be  referenced  in  any  way  In  the  SCTR  or  the 
PCTR. 

The  SCTR,  PCTR,  PICS,  and  PIXIT  are  the  official  record  of  the  product  test. 
The  Registration  Authority  application  form  is  for  administrative  purposes  and 
registration  clarification.  The  application  is  not  used  or  referenced  relative  to  the 
product  assessment,  nor  provided  to  acquisition  authorities.  In  other  words,  the 
SCTR,  PCTR,  PICS,  and  PIXIT,  not  the  application,  represent  the  testing  of  the 
products'  functionality  and,  by  themselves,  should  provide  sufficient  detail  that 
the  test  can  be  reconstructed  and  consistently  repeated,  at  any  time,  with  the 
same  results.  This  makes  It  mandatory  that  these  documents  be  accurate, 
complete,  and  provide  the  details  of  the  test  configuration  (hardware,  software, 
etc.)  and  test  results. 
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The  application,  which  is  not  part  of  the  permanent  record,  does  not  supplant  nor 
supersede  the  SCTR,  PCTR,  PICS,  or  PIXIT.  The  information  provided  by  the 
application  may  be  used  to  clarify  the  form  which  the  Register  entry  may  take, 
but  will  not  modify  any  information  or  claims  found  in  the  SCTR,  PCTR,  or  PICS. 
The  product  assessment  is  based  solely  upon  the  information  in  the  SCTR, 
PCTR,  and  PICS.  It  is  the  policy  of  the  Registration  Authority  not  to  assume  or 
to  guess  the  details  of  the  testing  or  product  configuration.  If  the  required 
information  is  not  in  the  respective  documents  then  the  application  stands  to  be 
rejected. 

The  PICS  and  PIXIT  provide  the  declaration  of  the  product's  abilities  and  a 
specification  of  certain  software  configurations  which  need  to  map  to  the  actual 
test  case  selections.  If  a capability  Is  claimed  in  the  PICS  then  it  shall  be  tested 
unless  a disclaimer  appears  in  either  the  PICS  or  PCTR  stipulating  that  the 
capability  was  not  to  be  tested  and  the  reason  why  (e.g.,  future  capability, 
current  capability  not  being  claimed,  CLNP  over  LAN  instead  of  X.25,  ATS/ETS 
not  available). 

Manual  verdict  assignments  must  be  justified  In  the  observation  section  of  the 
PCTR.  All  observations  must  be  both  interpretable  and  meaningful  plus  provide 
adequate  evidence  to  substantiate  whatever  claim  is  being  made.  Copies  of  the 
test  log  are  required. 

(3)  Test  Conduct  Adheres  to  the  Relevant  Standards 

The  form  and  format  of  the  testing  must  be  per  the  Agent  of  the  IGOSS  Panel* 
documented  procedures,  the  appropriate  Stable  Implementation  Agreements, 
ISO  9646,  the  appropriate  Protocol  Standard(s),  and  accepted  test  practices. 

(4)  Test  Determined  that  the  Product  Meets  the  Standards  (ISO,  SIA) 

The  test  must  completely  test  the  product's  functionality,  regardless  of  the  state 
of  the  ATS  and/or  MOT(s).  All  broken  or  defective  ATCs/ETCs  with 
workarounds  shall  be  executed  and  testing  shall  be  to  the  International 
Standard.  Deficiencies  in  the  ATS  or  MOT  are  not  a loophole  to  be  used  to 
avoid  testing  product  functionality.  This  extent  of  testing  is  the  responsibility  of 
the  product  vendor  and  is  In  keeping  with  the  philosophy  that  the  Registration 
Authority  represents  the  Buyer  of  the  US  IGOSS  products. 

The  test  results  shall  be  correct  with  all  manual  verdicts  fully  documented.  All 
verdict  changes  must  be  consistent  with  relevant  standards  and  be  justified  by 
the  Registration  Authority.  Every  ATC  must  be  accounted  for,  by  ATC  name,  as 
either  Deselected;  Selected,  but  Not  Run;  or  Run  with  a verdict  of  Pass,  Fail,  or 
Inconclusive.  The  Inconclusive  verdict  requires  sufficient  justification  for  its 
existence,  as  well. 
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(5)  Test  Laboratory  Demonstrated  Adequate  Quality  Assurance  Procedures 

All  test  laboratories  are  NVLAP  or  SCC  accredited  which  requires  that  they  have 
a living  Quality  Manual  and  Quality  Assurance  practices.  The  test  laboratory 
shall: 

(a)  Provide  Document  Tracking  such  that  the  SCTR,  PCTR,  PICS,  and  PIXIT 
have  unique  identifications  assigned  to  them,  which  are  consistent 
throughout  the  documentation. 

(b)  Provide  A PCTR  based  on  the  ATC  names  rather  than  the  ETC  names. 
The  ATS  is  relatively  constant  while  the  ETCs  vary  from  MOT  to  MOT. 

(c)  Provide  evidence  of  a QA  program  by  the  neatness  and  readability  of  the 
test  report  documents. 

(d)  Provide  Test  Case  Counts  that  match  the  total  of  ATCs  in  the  ATS  and 
the  verdicts.  The  Test  Case  Count  should  be  equal  to  the  number  of  test 
cases  De-selected  plus  test  cases  selected,  but  not  run  plus  test  cases 
with  pass,  fail,  and  inconclusive  verdicts. 

(e)  Provide  Documents  and  Applications  that  avoid  the  assumption  of  the 
reader's  familiarity  with  the  product(s).  Product  Information  Specifications 
should  contain  the  following  information  in  a clear  and  understandable 
format  such  as: 

Vendor  Name: 

Product  Name: 

Version/Release: 

Operating  System: 

Hardware  Platform: 

(f)  Ensure  that  relevant  entries  in  the  documents  match  those  that  are  in  the 
IGOSS  Register  (e.g.,  MOT  names.  Test  Laboratory  names  and 
addresses,  ATS  identification).  Include  the  NVLAP  or  SCC  number  when 
referencing  the  test  laboratory. 

(g)  Ensure  completeness  of  the  documentation  such  that  all  the  Information 
required  to  assure  product  description  and  test  reproducibility  Is  provided. 
The  following,  information  relative  to  the  test  environment,  should  be 
contained  In  the  SCTR  and/or  the  PCTR. 

Protocol(s)  Tested: 

Test  System  Name: 

Test  System  Version  Number: 

Connectivity:  (e.g.,  802.2/802.3  or  1984  X.25  PLP/HDLC  Lap  B,  RS-232) 
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Dates  of  Testing: 

MOT  Release  Date: 

(h)  Ensure  consistency  of  information  throughout  the  documents  such  that 
there  are  no  contradictions  or  discrepancies. 

(i)  Ensure  that  the  Underlying  Stack  is  on  the  current  US  IGOSS  Register 
and  that  it  is  completely  specified  in  the  SCTR  and/or  PCTR.  This 
includes  the  following  Information  In  a clear  and  understandable  format 
such  as: 

Connectivity:  (e.g.,  802.2/802.3  or  1984  X.25  PLP/HDLC  Lap  B,  etc.) 
Product  Name:  (i.e.  name  or  identification  of  the  software  product  or 
package  that  implements  the  underlying  stack) 

Op  Sys:  (i.e.  operating  system  that  the  named  Underlying  Stack  product 
executes  under) 

Hardware:  (i.e.,  specific  name  and/or  detailed  description  of  the 
communication  board  or  hardware  needed  for  connectivity) 

NOTE:  When  the  Underlying  Stack  is  a LAN  it  is  not  necessary  that  It  be 
registered  In  order  to  claim  IGOSS  conformance,  but  it  Is  still  mandatory 
that  It  be  accurately  and  completely  identified  with  this  information. 

(j)  Ensure  that  1 00%  of  mandatory  test  cases  were  selected. 

(k)  Ensure  that  100%  of  the  optional  test  cases  were  selected  for 
implemented  optional  functions. 

(l)  Ensure  that  all  test  cases  that  were  selected,  but  not  run  are  justified  by 
the  registration  authority. 

(m)  Ensure  that  all  new  ATC/ETC  problems  are  submitted  to  the  Registration 
Authority  for  review  so  that  a decision  can  be  made  prior  to  the  application 
for  registration. 

(n)  Ensure  that  any  test  cases  that  are  de-selected  are  justified  as  for  either 
optional  functions  which  are  not  implemented  or  for  non-applicable  test 
cases. 

(o)  Ensure  that  everything  claimed  in  the  PICS  is  tested  or  that  a disclaimer 
exists  that  specifies  that  the  capability  is  not  being  claimed  and  why. 

(p)  Ensure  that  the  final  verdicts  are  clearly  displayed  In  the  PCTR. 

(q)  Ensure  that  all  of  the  verdicts  presented  in  the  SCTR  and  PCTR  fall  within 
the  allowed  classifications:  (1)  Pass;  (2)  Selected,  but  not  run;  (3) 
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Inconclusive;  (4)  De-selected;  or  (5)  Failed.  No  other  verdicts  are 
recognized  by  the  IGOSS  Testing  Program. 

(r)  Ensure  that  there  are  no  failure  verdicts  recorded.  The  lUT  must  pass 
every  applicable  test. 

(s)  Ensure  that  there  are  no  blank  verdicts  for  those  test  cases  that  were 
selected  and  run. 

(t)  Ensure  that  no  test  cases  were  run  that  do  not  appear  in  the  registered 
ATS. 

(6)  Minimization  of  Assessment  Costs  and  Duration 

In  order  to  maintain  or  reduce  the  cost  of  doing  product  assessments  and 

registrations  as  well  as  reducing  the  time  required  to  perform  them,  the  following 

should  be  done: 

(a)  Provide  high-quality  test  reports. 

(b)  Provide  complete  explanations  per  observations  or  justifications.  If  the 
Registering  Authority  finds  it  necessary  to  do  the  research  then  the 
assessment  time  and  cost  will  increase. 

(c)  Provide  all  of  the  required  information.  The  more  information  and  details 
provided  by  the  vendor,  the  less  time  the  Registration  Authority  has  to 
devote  to  research  or  contacting  the  applicant  for  information. 

(d)  DO  NOT  attempt  to  contact  and/or  badger  the  Registration  Authority 
personnel  regarding  the  assessment's  progress.  Frequent  inquiries  into 
the  status  of  a registration  only  results  in  the  prolonging  of  the 
assessment.  The  Registration  Authority  will  do  product  assessment  on  a 
first  come  (i.e.,  payment  received),  first  serve  basis  and  will  call  the 
vendor  regarding  any  questions  or  Issues.  Plan  well  ahead  on 
submissions  so  as  to  provide  sufficient  time  before  deadlines  to  handle 
any  submission  discrepancies  that  might  occur.  Failure  to  plan  on  the 
part  of  the  vendor  does  not  constitute  a crisis  on  the  part  of  the 
registration  authority. 
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ANNEX  B:  MEANS  OF  TESTING  VALIDATION 


Note:  The  technical  information  concerned  with  the  Means  of  Testing  validation 
methodology  presented  within  this  Annex  is  provided  for  information  only.  The 
actual  reference  for  the  IGOSS  Means  of  Testing  Validation  Methodology  is  the 
latest  approved  version  of  [JITC-1]. 


B.O  .Tutorial 

The  IGOSS  Means  of  Testing  Assessment  and  Registration  requirements  are  being 
changed  to  reflect  needed  improvements  in  test  system  quality  and  to  formalize  some 
practices  which  have  developed  in  the  day  to  day  management  of  the  register.  The 
motivation  for  the  changes  is  to  enable  more  rigorous  product  testing  standards,  within 
the  limit  of  the  state  of  the  art. 

When  NIST/CSL  first  established  the  GOSIP  Testing  Program  in  1990,  the  state  of 
MOT  technology  was  quite  variable,  with  some  systems  achieving  high  quality  and  a 
high  percentage  of  test  suite  coverage,  and  other  systems  with  many  outstanding 
software  problems  and  a relatively  lower  level  of  test  suite  coverage.  The  Abstract  Test 
Suites  upon  which  coverage  was  assessed  varied  in  quality  and  coverage.  At  that  time 
we  deliberately  chose  to  accept  MOTs  into  the  program  having  a minimum  coverage  of 
60%  of  the  available  ATS. 

The  situation  today,  in  1993,  is  that  there  is  now  a well  established  MOT  registration 
program,  for  all  types  of  GOSIP  Version  2 protocols.  Many  problems  have  been  found 
in  the  initially  registered  Abstract  Test  Suites,  and  we  are  ready  to  accept  improved 
ATS  for  many  protocols  from  the  Open  Systems  Testing  Consortium  (OSTC)  who 
remain  the  principal  support  for  test  suites  which  are  tailorable  to  U.S.  GOSIP 
requirements.  It  is  now  appropriate  to  establish  the  MOT  assessment  program  for 
IGOSS  Version  1 MOTs.  Test  suites  for  some  of  the  Version  1 protocols,  such  as  X.25 
Packet  and  HDLC  LAP  B,  are  at  the  IS/DIS  levels  In  ISO.  MOTs  based  on  these  tests 
may  be  fully  registered,  assuming  at  least  95%  coverage. 


Other  changes  to  the  assessment  and  registration  program  include  the  following: 

1 Any  MOT  operating  over  an  X.25  platform  must  be  able  to  interoperate  with  lUTs 
which  implement  the  1984  standard  capabilities.  This  is  in  support  of  the 
requirement  that  an  lUT  operating  over  X.25  must  be  able  to  demonstrate  at 
least  Basic  Interconnection  in  order  to  be  registered  for  that  platform.  It  is 
accepted  that  this  does  not  necessarily  Imply  that  the  MOT  itself  must  support 
X.25  (1984).  It  would  be  inappropriate  to  make  that  requirement  since  many 
Wide  Area  Networks  do  not  support  1984  capabilities. 
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2 The  ability  for  MOT  suppliers  to  provide  minor  enhancements  or  bug  fixes  which 
do  not  change  the  test  case  coverage  without  requiring  a full  assessment. 

3 The  ability  for  MOT  suppliers  to  register  incremental  improvements  in  test  case 
coverage  and  bug  fixes  without  a complete  Dynamic  Validation  by  the  MOT 
Validation  Laboratory.  The  MOT  supplier  must  provide  documentary  evidence  of 
regression  testing  and  copies  of  test  case  logs  for  additional  or  improved  test 
cases.  This  also  opens  the  door  for  recognition  of  the  results  of  an  equivalent 
(or  better)  MOT  Validation  organization  - subject  to  the  establishment  of  a 
Memorandum  of  Understanding. 

4 A complete  dynamic  assessment  which  requires  exercise  and  scrutiny  of  100% 
of  those  test  cases  which  are  relevant  to  GOSIP,  in  the  registered  Abstract  Test 
Suite. 

5 Points  2-4  are  also  predicated  on  the  requirement  to  show  proof  that  the 
changes  are  reflected  In  Improved  product  documentation. 

These  changes  are  not  without  cost  - particularly  the  move  to  100%  validation.  It  is 
reasonable  for  the  cost  of  a full  MOT  validation  to  be  more  than  the  cost  of  a 
commercial  conformance  testing  campaign,  since  the  effort  and  the  skills  involved  are 
considerably  greater.  This  cost  is  mitigated  by  the  three  tiered  structure  of  the 
assessments  and  registrations  involving  (1)  nominal  change,  (2)  static  assessment  and 
(3)  full  dynamic  assessments  for  which  the  fees  will  also  be  tiered  accordingly.  Actual 
responsibility  for  determining  the  fees  belongs  to  the  Approved  MOT  Validation 
Laboratories. 


B.1.  Introduction 

The  program  of  MOT  assessments  and  registration  is  administered  by  the  Agent  of  the 
IGOSS  Panel*  who  has  authority  delegated  by  the  executive  director  of  the  IGOSS 
Panel.  The  accepted  means  of  testing  validation  methodology  for  the  IGOSS  is  the 
most  updated  version  of  [JITC  1]. 

B.1.1  Purpose 

This  Annex  explains  the  technical  and  administrative  procedures  for  the  assessment  of 
MOTs  for  GOSIP  protocols.  The  assessments  are  conducted  by  technical  experts. 
MOTs  which  are  found  to  be  valid  are  registered  by  the  Agent  of  the  IGOSS  Panel*  and 
for  use  in  Conformance  Test  Laboratories  which  are  seeking  NVLAP  or  SCC 
Accreditation. 

In  certain  circumstances,  MOTs  showing  some  deficiencies  from  the  requirements  may 
be  provisionally  registered  - and  recommended  for  Conformance  Test  Laboratory  use  - 
pending  rectification  of  the  deficiencies. 


Version  1.0 


June  1,  1994 


Page  62 


Annex  B:  Means  of  Testing  Validation 


B.1.2  Method 

ISO  IS  9646  OSI  Conformance  Testing  Methodology  and  Framework  describes 
methods  and  a process  by  which  implementations  or  partial  implementations  of  7-layer 
and  3-layer  OSI  stacks  may  be  tested  for  conformance  to  the  base  protocols.  The 
realization  of  test  methods  as  Means  of  Testing  generates  a further  testing  problem:  by 
what  method  shall  a Means  of  Testing  itself  be  validated?  A framework  for  the 
execution  and  interpretation  of  results  is  required.  The  problem  is  compounded  by  the 
fact  that  no  Reference  Implementation  can  exist  without  comprehensive  and  systematic 
testing,  which  itself  cannot  occur  without  a Means  of  Testing. 

The  solution  adopted  for  the  GOSIP  Testing  program  is  a bootstrap  method.  The  initial 
candidate  Reference  Implementation  is  executed  against  the  initial  candidate  MOT, 

using  the  process  defined  In  9646-5,  and  the  results  interpreted  in  t^  ways: 

1)  conventionally,  to  assess  the  candidate  Implementation  Under  Test  (lUT) 
response,  the  results  are  checked  against  the  Abstract  Test  Suites. 

2)  the  MOT  Is  statically  checked  against  the  criteria  given  in  the  main  part  of  this 
document;  and  the  results  of  execution  are  used  to  assess  the  MOT's  encoding 
and  decoding  Integrity,  and  its  faithful  mapping  of  the  Executable  Test  Suite  with 
respect  to  the  Abstract  Test  Suite. 

After  the  candidate  Reference  Implementation  has  been  conformance  tested,  the 
known  bugs  are  catalogued.  If  it  meets  the  operational  criteria,  the  Reference  is 
provisionally  registered  (as  a "Reference  Implementation"  and  NOT  as  an  IGOSS 
conformance  tested  product,  since  in  fact  the  Reference  may  not  meet  the  criteria  for 
entry  onto  the  tested  products  register),  and  may  be  employed  in  the  MOT  assessment 
and  registration  program,  using  the  method  described  in  this  document. 

The  prerequisites  for  the  application  of  this  method  of  MOT  assessment  are  high- 
quality  Abstract  Test  Suites,  and  a sufficient  expertise  in  OSI  protocol  and  test 
technology. 

B.1.3  Field  of  Application 

This  Annex  Is  addressed  to: 

(1)  Suppliers  of  IGOSS  Means  of  Testing  wishing  to  achieve  Agent  of  the  IGOSS 
Panel*  registration. 

(2)  Suppliers  of  OSI  test  services  seeking  accreditation  as  an  IGOSS  Conformance 
Test  Laboratory,  and  requiring  to  use  the  Agent  of  the  IGOSS  Panel*  registered 
Means  of  Testing. 
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(3)  Technical  experts,  who  use  the  procedures  herein  to  conduct  MOT  assessment 
campaigns. 

(4)  Other  MOT  assessment  authorities  which  seek  to  enter  into  mutual  recognition 
and  common  registration  arrangements. 

B.2.  Summary  of  the  Assessment  process 

B.2.1  The  Summary 

(1)  The  Agent  of  the  IGOSS  Panel*  announces  the  availability  of  Standard 
Reference  Materials  (SRMs)  for  the  testing  of  particular  IGOSS  protocols.  SRMs 
include  Abstract  Test  Suites  (Internationally  standardized  or  not). 

(2)  The  MOT  supplier  applies  to  a registered  Validation  Laboratory  for  Information. 
The  Validation  Laboratory  supplies  application  forms,  reference  entity 
configuration  information  (if  available),  PIXIT  proforma  and  Information  about  the 
process  of  validation. 

(3)  The  MOT  supplier  completes  the  application  form  and  provides  to  the  Validation 
Laboratory:  ATS/ETS  mapping,  ETS  defect  list,  baseline  or  regression  test 
report,  user  documentation,  tester  PIXIT  information,  test  log  files  (In  the  case  of 
a version  update),  and  the  appropriate  fees. 

(4)  The  MOT  Validation  Laboratory  reviews  the  application  and  may  perform  a full 
static  review  of  the  MOT  supplier's  claims. 

(5)  If  the  MOT  passes  the  static  review  It  may  be  directly  registered.  In  the  case  of 
update  to  a previously  registered  MOT.  Otherwise  a complete  dynamic 
validation  is  necessary.  A date  and  a site  for  the  dynamic  validation  will  be 
negotiated. 

(6)  The  MOT  Validation  Laboratory  assessor  conducts  a dynamic  review  and  makes 
a report  to  the  MOT  supplier,  the  IGOSS  Registrar,  and  the  Agent  of  the  IGOSS 
Panel*. 

(7)  Based  on  the  results  of  the  validation  the  MOT  may  be  registered,  or  registration 
may  be  deferred  pending  corrections.  The  complete  list  of  registration 
possibilities  is  given  in  section  B.5.2  below. 

(8)  Candidate  IGOSS  Conformance  Test  Laboratories  which  are  seeking  NVLAP  or 
see  Accreditation  select  MOTs  from  the  IGOSS  Register.  Conformance  Test 
Laboratory  accreditation  and  registration  will  be  based  on  Registered  or 
Provisionally  Registered  MOTs  only. 
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B.2.2  Routes  to  Registration 
B.2.2.1  Tiers 

A three-tiered  assessment  scheme  has  been  established  for  MOTs  because  the 
review/assessment  effort  is  different  according  to  what  is  being  registered,  and  what 
level  of  preparation  the  MOT  supplier  has  applied  prior  to  application.  The  three  tiers 
therefore  reflect  costs  for  the  Validation  Laboratory  and  the  Registration  function,  and 
are; 

COMPLETE  Assessment 
PARTIAL  assessment 
NOMINAL  check 

The  Complete  Assessment  may  be  required  under  some  of  the  following  conditions: 
IGOSS  version  change; 

Abstract  Test  Suite  coverage  (%)  requirement  increase; 

Abstract  Test  Suite  change; 

Provisional  to  Full  registration  change; 

Insufficient  confidence  in  product  or  testing; 

Major  changes  in  connectivity. 

Four  processes  are  involved  in  the  complete  assessment: 

A)  Manual  review  of  test  reports  against  a baseline  IDT  supplied  by  the  vendor. 

B)  Full  Dynamic  Assessment  of  the  MOT  involving 

1)  A manual  review  of: 

a)  Claimed  coverage  of  ATS, 

b)  Mapping  of  Executable  Test  Cases  versus  ATS, 

c)  Mapping  of  PICS  to  test  case  selection. 

2)  Execution  of  all  ETC's. 

3)  Evaluation  of  the  test  logs  for  all  ETC's  Versus  ATS. 

4)  Assessment  of  coverage  of  ATS. 

C)  Manual  review  of  the  product  documentation  provided  by  the  vendor. 

D)  Manual  review  of  defect  reports  supplied  by  the  vendor. 
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The  Partial  Assessment  involves  the  manual  review  of  test  reports  supplied  by  the 
vendor  and  the  submissions  of  the  test  logs  from  new  test  cases  or  modified  test  cases 
of  the  MOT  against  the  baseline  lUT.  An  updated  defects  report  is  also  required.  A 
partial  assessment  would  be  required  for: 

A revision  for  a MOT  version  update; 

Platform  change; 

Mutual  Recognition  Arrangement; 

Transparent  changes  in  connectivity. 

The  Nominal  check  involves  a documentary  review,  including  evidence  that  the  change 
does  not  require  partial  or  complete  testing.  In  principle,  a nominal  check  is  for  a 
cosmetic  change  that  does  not  significantly  affect  the  test  execution  engine.  The 
vendor  must  send  a notification  and/or  application  plus  required  fees. 

B.2.2.2  Objects 

The  general  apprach  Is  that  objects  which  can  be  entered  onto  the  MOT  register,  and 
their  required  levels  of  assessment,  are  as  follows: 

1)  Base  Product  (Complete) 

A previously  unregistered  MOT,  or  any  MOT  which  warrants  a full  dynamic 
assessment. 

2)  Derived  Connectivity  (Complete) 

Derived  from  a currently  registered  Base  MOT,  based  on  change  of  supporting 
connectivity  that  requires  protocol  changes,  such  as  LAN  instead  of  WAN  for 
CLNP. 

3)  Unverified  Functional  Update  (Complete) 

An  MOT  change  which  involves  alteration  to  testing  functionality  or  Improved  test 
coverage  which  is  NOT  adequately  corroborated  by  supporting  test  log  files  and 
documentation. 

4)  Derived  Product  (Partial) 

Derived  from  a currently  registered  (Base)  MOT,  based  on  some  change  of 
hardware  or  operating  system. 

5)  Derived  Substack  (Partial) 
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Derived  from  a currently  registered  (Base)  MOT  based  on  change  of  supporting 
connectivity  that  does  not  alter  the  protocol's  behavior,  i.e.  TPO  instead  of  TP4 
under  FTAM. 

6)  Verified  Functional  Update  (Partial) 

An  MOT  change  which  involves  alteration  to  testing  functionality  or  improved  test 
coverage,  which  is  corroborated  by  supporting  test  log  files  and  documentation, 
i.e.  MOT  Version  update. 

7)  Mutual  Recognition  Arrangement  (Partial) 

An  MOT  registered  with  another  authority  recognized  by  the  Agent  of  the  IGOSS 
Panel*,  for  which  IGOSS  registration  Is  sought. 

8)  Cosmetic  Update  (Nominal) 

A version  change  which  involves  no  alteration  to  testing  functionality  or  the  test 
coverage,  and  a change  that  does  not  affect  the  protocol  engine,  i.e.  user 
interface  changes. 

B.3.  Static  Assessment 

B.3.1  Goals  of  Static  Assessment 

1)  To  determine  whether  an  MOT  claims  support  of  IGOSS  functionality  and 
provides  at  least  the  minimum  required  test  coverage. 

2)  To  determine  whether  the  user  documentation  adequately  describes  Installation 
of  the  MOT,  Selection  , parameterization,  execution,  results  analysis  and  verdict 
assignment  for  each  test  case;  also,  that  the  documentation  describes  the 
correspondence  between  executable  test  cases  and  the  registered  Abstracts, 
and  that  all  test  case  defects  are  fully  described. 

3)  To  determine  whether  a Partial  or  Complete  Dynamic  Assessment  Is  necessary. 

B.3.2  Process 

(1)  Check  that  all  forms  and  fees  are  supplied  and  completed. 

(2)  Check  the  Means  of  Testing  capabilities  against  the  generic  requirements  in 
Appendix  III  of  this  document. 
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(3)  Determine  the  coverage  of  the  Executable  Test  Suite  using  the  method 
described  in  B.3.3.2  below.  Determine  which  tests  are  claimed  as  defective,  and 
their  percentage  with  respect  to  the  total  number  of  tests. 

If  the  MOT  has  been  recognized  under  a Mutual  Recognition  Arrangement, 
determine  whether  its  coverage  is  valid  for  IGOSS. 

(4)  Determine  whether  It  is  appropriate  to  proceed  with  Partial  or  Complete  dynamic 
assessment. 

B.3.3  Test  Suite  Issues 

B.3.3.1  Abstract  Test  Suite  Validity 

Abstract  Test  Suites  are  selected  from  a variety  of  different  sources,  and  sometimes  at 
arbitrary  levels  of  coverage  and  quality.  These  circumstances  combine  to  determine 
the  'state  of  the  art'  for  the  precision  of  conformance  testing.  It  Is  desirable  to  have  this 
level  Increase  in  precision,  and  at  the  same  time  to  have  the  ATS  formalized  to  become 
International  Standards. 

The  mechanism  has  been  to  select  and  review  ATS  based  on  the  current  version  of 
IGOSS,  and  update  the  ATS  In  conjunction  with  successive  version  releases  of  IGOSS. 
The  limiting  factor  on  this  improvement  process  has  been  the  amount  of  effort  available 
to  fix  bugs  in  the  Abstract  Test  Cases  and  to  add  more  test  cases  to  improve  the 
coverage.  This  process  of  test  suite  management  will  probably  not  stop  even  after  the 
IS  level  has  been  reached  for  any  particular  test  suite  - since  the  standardization  of  a 
test  suite  has  no  bearing  on  its  validation  through  experience  of  use. 

B.3.3.2  Coverage  Guidelines 

In  order  to  assess  the  Test  Suite  coverage  for  the  MOT  there  are  two  possibilities: 

(I)  There  exists  a Registered  Abstract  Test  Suite  for  the  MOT, 

(II)  There  does  not  exist  a Registered  Abstract  Test  Suite  for  the  MOT. 

In  the  case  where  a Registered  Abstract  Test  Suite  exists,  method  (I)  must  be  chosen; 
otherwise,  method  (II)  must  be  chosen. 

(I)  Compare,  or  cross  reference  the  Executable  Test  Suite  against  the  Abstract  Test 
Suite  to  establish  coverage.  In  terms  of  breadth  and  depth. 

Breadth:  there  shall  be  one  Executable  Test  Case  corresponding  to  each  IGOSS 
function  in  the  PICS.  M IGOSS  functionality  must  be  testable. 

Depth:  there  shall  be  one  Executable  Test  Case  for  each  function  tested  by  the 
Abstract  Test  Suite.  Note,  this  does  not  require  a one  to  one  correspondence 
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between  ETCs  and  ATCs:  one  Executable  Test  Case  may  be  implemented  to 

realize  more  than  one  test  purpose. 

(II)  Using  a IGOSS  PICS  as  a basis,  apply  the  coverage  criteria  given  In  ISO  IS 

9646-2,  section  10.4,  and  reproduced  here: 

(a)  for  Capability  test  groups: 

(1)  at  least  one  test  purpose  per  relevant  capability; 

(2)  at  least  one  test  purpose  per  relevant  PDU  type  and  each  major 
variation  of  each  such  type,  using  normal  or  'default'  values  for 
each  parameter; 

(b)  for  test  groups  concerned  with  test  event  variation  in  each  state:  at  least 
one  test  purpose  per  relevant  state/event  combination; 

(c)  for  test  groups  concerned  with  timers  and  timing,  at  least  one  test  purpose 
concerned  with  the  expiry  of  each  defined  timer; 

(d)  for  test  groups  concerned  with  encoding  variations,  at  least  one  test 
purpose  for  each  relevant  kind  of  encoding  variation  per  relevant  PDU 
type; 

(e)  for  test  groups  concerned  with  valid  individual  parameter  values: 

(1)  for  each  relevant  integer  parameter,  test  purposes  concerned  with 
the  boundary  values  and  one  randomly  selected  mid-range  value; 

(2)  for  each  relevant  bit-wise  parameter,  test  at  least  one  value  which 
is  not  the  'default'  value; 

(3)  for  other  relevant  parameters,  at  least  one  test  purpose  concerned 
with  a value  different  from  what  Is  considered  'normal'  or  default  In 
other  test  groups; 

(f)  for  test  groups  concerned  with  invalid  individual  parameter  values: 

(1)  for  each  relevant  integer  parameter,  test  purposes  concerned  with 
invalid  values  adjacent  to  the  allowed  boundary  values  plus  one 
other  randomly  selected  Invalid  value; 

(2)  for  each  relevant  bit-wise  parameter,  test  purposes  for  as  many 
invalid  values  as  practical; 
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(3)  for  all  other  relevant  types  of  parameter,  at  least  one  test  purpose 
per  parameter; 

(g)  for  test  groups  concerned  with  combinations  of  parameter  values: 

(1)  at  least  one  test  purpose  for  each  important  combination  of  specific 
values  (e.g.,  boundary  values); 

(2)  at  least  one  test  purpose  per  set  of  Interrelated  parameters  to  test 
an  'unusual'  combination  of  relevant  values." 


B.4.  Dynamic  Validation 

This  section  provides  the  procedures  for  conducting  a dynamic  review  of  a IGOSS 

MOT.  Since  the  intention  of  this  assessment  procedure  Is  to  determine  compliance  of 

MOTs  with  IGOSS  testing  requirements,  there  will  be  no  extraneous  diagnostic 

assessment  or  recommendations.  Assessors  are  directed  not  to  inquire  too  deeply  into 

the  underlying  reasons  behind  non-compliance;  it  is  sufficient  to  note  the  occurrence. 

An  MOT  supplier  may  opt  to  negotiate  exit  from  the  assessment  process  at  any  time. 

B.4.1  Goals  of  Dynamic  Validation 

1)  To  develop  confidence  that  an  MOT  provides  test  coverage  at  or  above  the 
current  threshold  level,  and  to  document  defects  in  coverage  so  as  to  allow 
accurate  review  of  test  reports  prior  to  product  registration. 

2)  To  corroborate  a claim  of  improved  test  coverage. 

B.4.2  Prerequisites 

B.4.2.1  Validation  Laboratory  Responsibilities 

1)  that  a Reference,  or  known  implementation,  be  installed  and  operational  on  a 
separate  system.  This  shall  be  under  the  control  of  the  MOT  Validation 
Laboratory.  It  is  preferable  that  the  Validation  Laboratory  has  tested  Reference 
Implementations,  but  there  is  the  special  case  of  a first  time  MOT,  or  the  case 
where  a suitable  Reference  is  not  available  to  the  Validation  Laboratory. 

2)  that  the  Registered  Abstract  Test  Suite  be  available  for  validation  of  results.  This 
will  be  used  in  conjunction  with  the  Abstract  Test  Case  defects  list. 

B.4.2.2  MOT  Supplier  Responsibilities 

1)  that  the  MOT,  including  all  associated  tools,  be  installed  on  a system  and  be 
operational; 
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2)  that  the  MOT  supplier  provides  a conformance  log  and  SCTR/PCTR  for  an  lUT 
of  their  choice,  to  be  used  as  a baseline  for  the  assessment. 

B.4.2.3  Shared  Responsibilities 

1)  that  the  static  analysis  of  the  Reference  and  of  the  MOT  has  been  performed 
prior  to  the  assessment. 
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B.4.3  Dynamic  Assessment  Procedures 
B.4.3.1  Complete  Assessment 

(1)  Establish  Basic  Interconnection  between  MOT  and  System  Under  Test. 

(2)  Conduct  a review  of  the  MOT  documentation,  to  determine  whether  an  MOT  can 
be  configured  and  operated  without  direct  assistance  from  the  MOT  supplier. 

(3)  Validate  the  characteristics  of  the  MOT  from  generic  characteristics  given  below. 
For  each  specified  characteristic,  select  and  execute  a test  and  assess  the 
MOTs  support  of  that  characteristic.  For  Instance,  in  CLNP,  the  characteristic 
"capability  to  construct  CLNPDUs  according  to  both  the  fully  segmenting  and  the 
non-segmenting  subsets,  and  send  them  to  a CLNP  end  system  under  test  over 
any  supported  medium".  Select  tests  which  involve  the  generation  of  fully 
segmented  and  non-segmented  CLNPDUs.  Using  log  file  analysis  tools  and  IS 
8473,  determine  that  the  format  and  content  of  one  sample  of  each  CLNPDU  is 
correct. 

Another  example:  in  FTAM,  the  "capability  to  negotiate  and  encode  specified 
document  types".  Select  and  execute  a test  which  includes  the  use  of  the 
document  type  FTAM-3.  Using  log  file  analysis  tools  and  IS  8571-2,  determine 
that  the  format  and  content  of  each  FADU  is  correct. 

(4)  Using  the  Static  Assessment  results  from  Section  B.3,  above,  validate  the 
Executable  Test  Suite  for  the  MOT.  There  are  two  possibilities  for  this 
procedure: 

(I)  There  exists  a Registered  Abstract  Test  Suite  for  the  MOT: 
demonstration  of  the  equivalence  of:  the  results  of  execution  of  an 
Executable  Test  Case,  and  one  path  through  each  associated  Abstract 

Test  Case  (if  more  than  one).  Note,  M tests  are  executed.  Independently 
of  the  supplier's  claims,  since  this  is  primarily  a test  of  the  MOTs 
capabilities  and  not  those  of  the  System  Under  Test. 

(II)  There  does  not  exist  a Registered  Abstract  Test  Suite  for  the  MOT: 
validate  the  results  of  execution  of  each  Executable  Test  Case,  by 
comparison  against  the  base  standard  and  relevant  Implementor's 

agreements.  Note,  M tests  are  executed,  independently  of  the  supplier's 
claims,  since  this  is  primarily  a test  of  the  MOTs  capabilities  and  not  those 
of  the  System  Under  Test. 

Generic  MOT  Characteristics 
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1)  The  capability  to  analyze  PICS  and  PIXIT  for  the  IDT  and  to  select  and 
parameterize  tests  to  be  run,  and  to  configure  the  means  of  testing  for 
communication  with  the  SUT. 

2)  Procedures  to  reconcile  PDU  and  test  data  with  the  test  purposes  and  yield  a 
verdict  for  each  test  purpose. 

3)  The  capability  to  produce  Conformance  Test  Reports,  listing  test  cases  executed 
and  their  verdicts,  and  detailing  the  lUT  behavior  in  cases  of  failure. 

4)  Capability  to  record  the  protocol  data  units  exchanged  with  the  lUT  in  a 
conformance  log,  and  to  review  the  structure  and  encoding  of  protocol  data  units 
after  the  test  campaign  is  complete. 

5)  The  capability  to  construct  PDUs  for  the  appropriate  protocol  and  send  them  to 
an  attached  lUT. 

6)  The  capability  to  receive  and  decode  PDUs  according  to  the  appropriate  protocol 
specification.  The  capability  to  validate  PDUs  received  and  record  the  results  In 
a conformance  log. 

7)  The  capability  to  construct  invalid  as  well  as  valid  PDUs. 

8)  The  capability  to  monitor  and  Initiate  exchanges  of  PDUs  between  the  Means  of 
Testing  and  the  lUT  (Inopportune  as  well  as  normal). 

B.4.3.2  Partial  Assessment 

(1)  Establish  Basic  Interconnection  between  MOT  and  System  Under  Test. 

(2)  Using  the  Static  Assessment  results  from  section  B.3,  above,  for  every  (formerly 
defective)  test  case  claimed  as  fixed,  validate  the  test  case  using  one  of  the 
following  two  methods: 

(I)  If  there  exists  a Registered  Abstract  Test  Suite  for  the  MOT:  demonstrate 
the  equivalence  of  the  results  of  execution  of  an  Executable  Test  Case, 
and  one  path  through  each  associated  Abstract  Test  Case  (if  more  than 
one). 

(II)  If  there  does  not  exist  a Registered  Abstract  Test  Suite  for  the  MOT: 
validate  the  results  of  execution  of  each  Executable  Test  Case,  by 
comparison  against  the  base  standard  and  relevant  Implementor's 
agreements. 

B.4.4  Assessor's  Report 
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After  completion  of  the  static  and  dynamic  aspects  of  the  Assessment,  the  assessor 
completes  the  MOT  Qualification  Report,  one  copy  of  which  goes  to  the  MOT  supplier 
and  one  copy  to  MOT  Registration  Authority.  The  MOT  supplier  is  required  to  provide  a 
non-altered  copy  of  the  MOT  Registration  Report  to  all  its  NVLAP  and  SCO  clients. 

The  completed  assessor's  report  consists  of: 

The  results  of  Static  Assessment, 

The  results  of  Dynamic  Assessment, 

The  corroborated  list  of  test  case  defects. 

The  calculation  of  percentage  coverage. 

The  assessor's  recommendation  for  registration. 

The  MOT  supplier  may  choose  to  disagree  with  any  portion  of  the  report  by  appending 
his  signed  comments  indicating  the  contested  portion(s).  This  appeal  may  be  made 
directly  to  NIST,  who  will  inquire  Into  the  reasons  for  the  disagreement  and  may 
corroborate  or  overturn  the  Validation  Laboratory's  decision.  Technical  issues  may  be 
referred  to  the  OIW  for  their  consideration. 

B.5.  Registration 
B.5.1  Fee  Basis 

Fees  are  charged  for  MOT  Validation  and  Registration  based  on  the  work  required  to 
determine  an  MOT's  validity.  This  Is  based  on  the  "Objects"  of  section  B.2.2.2. 

B.5.2  Categories  and  Criteria  for  Registration 

The  following  actions  may  be  taken  by  the  MOT  Registrar: 

(1)  Registration  of  the  product 

(2)  Provisional  Registration  of  the  product 

(3)  Deferral  of  Registration  pending  corrections 

(4)  Denial  of  Registration 

(5)  Revocation  of  Registration 

(6)  Expiration  of  Registration 

(7)  Withdrawal  by  the  MOT  Supplier. 

The  criteria  for  assignment  to  each  category  are: 

(1)  Registration  of  the  product 

(a)  operation  of  the  MOT  over  a Registered  IGOSS  platform,  I.e.,  that  part  of 
the  MOT  stack  which  provides  OSI  connectivity  shall  have  been 
successfully  tested  and  registered  as  a IGOSS  conformant  product 
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(Unless  valid  arguments  to  the  contrary  can  be  made  that  this  need  not  be 
the  case). 

(b)  correct  implementation  of  95%  or  more  of  the  Registered  Abstract  Test 
Suite(s).  Note,  the  ATS  must  be  fully  registered,  not  provisionally. 

Both  conditions  (a)  and  (b)  must  occur  before  registration  can  be  granted.  Registration 
is  granted  until  the  expiration  of  registration  of  the  associated  Abstract  Test  Suite  (or 
the  earliest  such  expiration  in  the  case  of  multi-layer  testers  tested  using  embedded 
methods),  or  until  withdrawal  of  the  product  by  the  MOT  supplier. 

(2)  Provisional  registration  of  the  product 

(c)  operation  of  the  MOT  over  a working  IGOSS  platform,  I.e.,  that  part  of  the 
MOT  stack  which  provides  OSI  connectivity  is  able  to  interoperate  with  the 
System  Under  Test  In  the  context  of  a test  campaign. 

(d)  correct  implementation  of  not  less  than  the  current  threshold  percentage 
of  the  Registered  Abstract  Test  Suite,  If  one  exists.  If  a Registered  ATS 
does  not  exist  then  the  determination  of  what  constitutes  ''100%"  is 
determined  by  the  Agent  of  the  IGOSS  Panel*,  using  the  ISO  guidelines 
given  In  B.3.3.(ll),  above  and  the  percentage  rule  Is  applied  to  this  figure. 
This  percentage  may  be  varied  from  year  to  year  to  stage  progressive 
improvement  in  the  Installed  testing  and  IGOSS  implementation  bases. 

(e)  Provisional  Registration  will  n^  occur  if  one  or  more  MOTs  for  the  same 
protocol  or  profile  are  already  fully  registered. 

If  conditions  (c)  and  (d)  occur  then  Provisional  Registration  may  be  granted. 
Provisional  Registration  is  granted  until  6 months  after  the  next  ATS  is  registered.  The 
MOT  supplier  may  apply  for  registration  based  on  an  improved  percentage  coverage  at 
any  time  during  the  currency  of  the  associated  ATS. 

(3)  Deferral  of  Registration 

If  agreement  has  been  reached  between  the  MOT  Validation  Laboratory  and  the  MOT 
supplier,  then  after  the  assessment  the  supplier  may  make  minor  changes  to  the  MOT 
which,  with  acceptable  documentary  evidence  of  the  fixes,  may  be  accepted  for 
deferred  registration. 

(4)  Denial  of  Registration 

Registration  may  be  denied  for  any  of  a variety  of  reasons,  including  the  following: 
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failure  of  the  MOT  to  interoperate  because  of  a defective  I GOSS  platform,  i.e., 
that  part  of  the  MOT  stack  which  shall  provide  connectivity  is  not  able  to 
interoperate  with  the  System  Under  Test. 

correct  implementation  of  less  than  the  current  threshold  percentage  of  the 
Registered  Abstract  Test  Suite. 

the  MOT  is  unable  to  execute  a test  campaign  without  crashing,  or  requires 
frequent  Initialization. 

product  documentation  is  Inadequate. 

the  list  of  known  defects  is  not  provided. 

(5)  Revocation  of  Registration 

If  an  MOTs  test  case  coverage  falls  below  the  threshold  percentage,  due  to  the 
discovery  of  executable  test  case  defects,  the  MOT  supplier  is  given  90  days  to  provide 
enough  fixes  to  ensure  that  the  threshold  percentage  is  exceeded.  If  within  90  days 
rectification  has  not  occurred,  as  demonstrated  by  a Static  Registration,  the  MOT  is 
removed  from  the  register.  After  this  time  no  products  tested  against  the  affected 
MOTA/erslon  will  be  entered  onto  the  Conformance  Tested  Products  Register. 

The  first  subsequent  version  of  the  same  MOT  shall  undergo  a complete  dynamic 
validation  before  entry  onto  the  register. 

(6)  Expiration  of  Registration 

When  an  MOT  Is  fully  registered  there  Is  normally  no  expiration  date  associated  with  it. 
This  Is  however  contingent  upon  the  stability  and  quality  of  the  associated  fully 
registered  ATS. 

In  the  provisional  situation,  Abstract  Test  Suites  are  periodically  updated  and 
registered,  and  MOTs  are  revalidated  and  registered  in  synchronization  with  these, 
usually  with  a higher  threshold  percentage  coverage  requirement.  In  cases  where  the 
registered  ATS  and  the  percentage  coverage  requirement  does  change,  existing 
registered  MOTs  shall  expire  6 months  after  the  registration  of  MOTs  based  on  the 
upgraded  coverage  requirement.  No  product  tested  against  an  expired  MOT  version 
will  be  entered  onto  the  Conformance  Tested  Products  Register. 

(7)  Withdrawal  by  the  MOT  Supplier 

An  MOT  version  which  is  registered  may  be  withdrawn  at  any  time  by  the  MOT 
Supplier.  After  this  time  no  products  tested  against  it  will  be  entered  onto  the 
Conformance  Tested  Products  register.  In  addition,  products  originally  tested  using  this 
MOT  which  are  entered  on  the  Conformance  Tested  Products  Register  may  be 
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annotated  with  an  indication  that  the  MOT  against  which  they  were  tested  has  been 
withdrawn. 

B.5.3  Derived  Implementation  Guidelines 

The  following  conditions  are  suggested  as  guidelines  for  determining  whether  a IGOSS 
Means  of  Testing  may  be  derived  from  a base  implementation: 

(a)  the  host  and  target  computer  systems  of  the  base  and  derived  IGOSS  MOTs 
have  compatible  instruction  sets  and  operating  systems.  Common  examples  of 
compatible  instruction  sets  and  operating  systems  are  two  different  computer 
system  models  in  a manufacturer's  product  line  or  the  computer  systems 
produced  by  different  manufacturers  that  use  the  same  hardware  mechanisms 
and  operating  systems. 

(b)  the  IGOSS  MOT  proposed  for  registration  was  derived  from  the  base 
Implementation  by  changes  that  are  within  the  scope  of  accepted  software 
maintenance  practices.  Arguments  along  this  line  should  be  included,  In  writing, 
with  the  application. 

(c)  The  PCTR  and  SCTR  for  the  IGOSS  MOT  are  either  the  same  as  the  base 
implementation  or,  if  there  are  minor  differences,  these  differences  are  justified 
as  being  within  the  scope  of  accepted  software  maintenance  practices. 

(d)  The  base  implementation  was  assessed  by  an  approved  and  registered 
laboratory  and  is  registered  with  the  Agent  of  the  IGOSS  Panel*. 

A derived  Implementation  may  lose  its  registration  if  it  is  challenged  successfully  (see 
B.6.2),  upon  expiration  of  the  Registration  of  the  base  implementation,  or  when 
Registration  is  revoked  by  the  MOT  supplier. 

The  Agent  Of  the  IGOSS  Panel*  will  not  perform  on-line  assessments  of  derived 
Implementations.  The  Agent  of  the  IGOSS  Panel*  will  review  the  application  for 
completeness  and  plausibility  of  information.  The  CSL  is  the  final  authority  on  whether, 
and  in  what  circumstances,  registration  shall  occur. 

Applications  which  are  acceptable  to  the  Agent  of  the  IGOSS  Panel*  are  entered  in  the 
register  of  IGOSS  MOTs  under  the  same  categorization  as  the  base  implementation, 
(I.e.,  Registered  or  Provisionally  Registered)  and  with  the  same  constraints  - 
Registration  is  based  on  the  expiration  of  the  Registered  Abstract  Test  Suite, 
Provisional  Registration  expires  at  the  same  time  as  that  of  the  base  implementation. 
The  MOT  supplier  will  be  notified  when  registration  is  approved.  The  register  will  be 
published  periodically,  and  will  be  available  on  request  from  the  Agent  of  the  IGOSS 
Panel*. 

B.5.4.  Additional  Guidance  for  MOT  Registrations 
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One  of  the  most  significant  activities  in  the  IGOSS  program  is  that  of  placing  products 
on  the  IGOSS  register.  This  requires  that  there  be  a method  by  which  the  products  can 
be  tested  for  conformance  to  the  relevant  standard(s).  This  testing  Is  realized  by  a 
protocol  specific  MOT  (Means  Of  Testing)  that  is  also  tested  and  placed  on  the  IGOSS 
register.  The  purpose  of  this  section  is  to  provide  some  level  of  guidance  for  the  MOT 
registration  other  than  Base. 

The  IGOSS  program  provides  two  types  of  MOT  registration;  base  and  derived.  The 
"base"  registration  is  the  registration  of  a MOT  that  has  been  fully  tested  by  an 
approved  accredited  laboratory.  This  testing  consists  of  an  extensive  evaluation  of 
both  the  MOT/user  interface  and  Test  Execution.  As  part  of  the  base  registration  the 
MOT  vendor  is  also  required  to  provide  a "Baseline  lUT  PCTR",  which  is  a Test  Report, 
generated  from  the  execution  of  the  MOT  at  the  Vendor's  site  against  a Reference 
Implementation  or  Known  Sample.  This  test  report  is  used  to  verify  that  any  MOT 
derived  from  this  base  MOT  is  able  to  execute  the  test  at  least  as  well  as  the  base 
MOT. 

A derived  MOT  registration  is  the  registration  of  a MOT  that  is  a modification  of  a 
"base"  registered  MOT. 

B.5.4.1  Definitions 

Test  Engine:  That  portion  of  the  MOT  that  executes  the  test  and  records  the  actions  of 
the  lUT  via  PDUs  exchanged  with  the  SUT. 

Virtual  Machine:  The  environment  within  which  the  Test  Engine  operates  which 
consists  of  the  hardware  platform  and  operating  system. 

User  Interface:  That  portion  of  the  MOT  that  provides  the  test  engineer  the  capability  to 
interact  with  and  manipulate  the  Test  Engine  and  other  MOT  functions.  The  User 
Interface  permits  the  test  engineer  to  perform  ETS  parameterization,  execute  the  ETS, 
and  analyze  the  log  files.  It  may  also  provide  for  test  case  evaluations,  verdicts,  log  file 
manipulation,  or  any  other  useful  activity. 

Connectivity:  The  connectivity  identifies  the  protocols  that  connect  the  Test  Engine  to 
the  lUT.  Connectivity  is  the  abstract  specification  of  the  supporting  stack  in  terms  of 
protocols. 

Underlying  Stack:  The  underlying  stack  Is  the  specific  product(s)  that  provide  the 
connectivity.  The  Underlying  Stack  is  specified  In  terms  of  the  software  and/or 
hardware  products  that  provide  the  connectivity. 

Baseline  lUT:  The  Baseline  lUT  is  a Reference  Implementation  or  Known  Sample  that 
Is  easily  accessible  to  the  MOT  vendor  for  purposes  of  exercising  the  MOT  at  the 
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"factory".  The  Baseline  lUT  must  be  capable  of  exercising  the  entire  ETS,  but  does  not 
have  to  be  capable  of  passing  each  test. 

In-House  Testing:  Testing  that  the  MOT  vendor  conducts  against  the  "Baseline  lUT"  at 
a location  convenient  to  the  vendor  other  than  an  approved  laboratory. 

Baseline  lUT  Test  Report:  The  Baseline  lUT  Test  Report  is  generated  by  the  In-House 
testing  campaign  and  provides  a record  of  the  MOT's  behavior  relative  to  the  selected 
Baseline  lUT. 

Registrations  Relative  to  the  Virtual  Machine:  Changes  to  only  the  operating  system 
(e.g.,  DOS  to  UNIX),  system  calls,  or  the  platform  are  classified  as  derived 
Registrations. 

B.5.4.2  Generic  Guidance 

This  section  presents  different  type  of  guidance  depending  upon  examples  of 
modification  to  the  base  MOT 

(0)  Derived  registrations  will  be  allowed  for  hardware  (only)  platform  changes  with 
conclusive  evidence  that  the  Test  Engine  remains  unchanged  due  to  being 
isolated  from  the  platform  architecture  by  the  operating  system.  This  Includes 
both  architecture  changes  and  hardware  platform  clones. 

Example: 

MOT/POSIX/platformX  ==>  MOT/POSIX/platformY 
MOT/OpSys1/platformX  ==>  MOT/OpSys1 /platform Y 
MOT/OPSys1 /platform  ==>  MOT/OPSys1 /platform  clone 

(1)  Testing  Requirement:  Full  In-House  Test  (i.e.,  all  ATCs) 

Derived  registrations  will  be  allowed,  without  testing,  for  hardware  platform 
changes  if  there  is  conclusive  evidence  presented  that  the  Test  Engine  remains 
unchanged. 

Example: 

MOT/OpSys/platformX  ==>  MOT/OpSys/platformX' 

MOT/OpSys/platformX  ==>  MOT/OpSys/platformY 

(2)  Testing  Requirement:  Full  In-House  Test  (i.e.,  all  ATCs) 

Derived  registrations  will  be  considered  if  there  is  conclusive  evidence  presented 
by  the  applicant  that  the  modifications  made  in  the  operating  system  were 
transparent  to  the  Test  Engine. 

Example: 
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MOT/OpSysX/platform  ==>  MOT/OpSysX'/platform 

(3)  Testing  Requirement:  Full  In-House  Test  (i.e.,  all  ATCs) 

Changing  the  virtual  machine  by  altering  both  the  hardware  and  the  software  is 
also  acceptable  for  a Derived  registration  and  requires  the  same  level  of  MOT 
testing  as  an  individual  change. 

Example: 

MOT/OpSys1/platformX  ==>  MOT/OpSys2/platformY 

(4)  Testing  Requirement:  Full  In-House  Test  (i.e.,  all  ATCs) 

Registrations  Relative  to  Connectivity:  Changes  In  connectivity  are  allowed  as  a 
Derived  Registration  if  there  Is  conclusive  evidence  presented  by  the  applicant 
that  the  modifications  made  in  the  operating  system  were  transparent  to  the  Test 
Engine.  The  connectivity  change  must  also  result  in  a legitimate  IGOSS 
configuration.  The  key  factor  to  this  registration  is  the  MOT's  continued  ability  to 
Interoperate  with  the  I LIT. 

Example: 

MOT/CLNP/LAN  ==>  MOT/CLNP/X.25 

(5)  Testing  Requirement:  Full  In-House  Test  (i.e.,  all  ATCs) 

Registrations  Relative  to  Underlying  Stack  Interface  Changes:  Derived 
Registrations  will  also  be  allowed  where  the  Underlying  Stack  is  changed,  but 
the  protocol  connectivity  remains  the  same. 

Example: 

MOT/CLNP-productX/U\N  ==>  MOT/CLNP-product  Y/LAN 

(6)  Testing  Requirement:  Full  In-House  Test  (i.e.,  all  ATCs) 

MOT  Registrations  Relative  to  User  Interface  Changes  - Because  the  Test 
Engine  is  insulated  from  most  User  Interface  changes.  It  Is  possible  to  Derived 
Register  a MOT  derived  from  a conformant  base  MOT  given  that  the  Test 
Engine  is  unchanged. 

Example: 

User  Interface1/M0T  ==>  User  Interface2/M0T 

(7)  Testing  Requirement:  Full  In-House  Test  (I.e.,  all  ATCs) 
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MOT  Registration  Relative  to  Test  Case  Correction  - A MOT  can  be  Derived 
Registered  based  on  a Base  MOT  when  one  or  more  test  cases  are  corrected, 
given  that  the  rest  of  the  Test  Engine  is  unaffected. 

Example: 

MOT(V1)/CLNP/LAN  ==>  MOT(V2)/CLN P/LAN 

(8)  Testing  Requirement:  Full  In-House  Test  plus  detailed  log  files  for  the  corrected 
ETC(s) 

MOT  Registration  Relative  to  Test  Case  Addition:  MOT  can  be  Derived 
Registered  from  a Base  MOT  when  one,  or  more,  test  cases  are  added  given 
that  the  rest  of  the  Test  Engine  Is  unaffected  and  the  new  ETCs  come  from  the 
registered  ATS  and  are  correctly  implemented. 

Example: 

MOT(V1)/CLNP/LAN  ==>  MOT(V2)/CLN P/LAN 

(9)  Testing  Requirement:  Full  In-House  Test  plus  detailed  log  files  for  the  modified 
ETC(s) 

MOT  Registration  Relative  to  Test  Case  Deletion:  A MOT  can  be  Derived 
Registered  from  a Base  MOT  if  the  Test  Engine  is  altered  to  affect  the  removal 
of  the  test  case  given  that  the  rest  of  the  Test  Engine  is  unaffected. 

Example: 

MOT(V1)/CLN P/LAN  ==>  MOT(V2)/CLN P/LAN 

(10)  Testing  Requirement:  Full  In-House  Test 

MOT  Registrations  Relative  to  Version  Changes:  Because  MOT  version  changes 
are  often  due  to  modifications  unrelated  to  the  Test  Engine,  Derived 
Registrations  will  be  allowed  for  a MOT  derived  from  a Base  MOT  given  that  the 
changes  do  not  affect  the  Test  Engine. 

Example: 

MOT(V1)/CLNP/LAN  ==>  MOT(V2)/CLN P/LAN 
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(11)  Testing  Requirement:  Full  In-House  Derived  Testing 

MOT  Registrations  Relative  to  Added  whole  ATSs:  A Derived  Registration  will  be 
allowed  for  any  MOT  that  adds  at  least  one  new  ATS  to  its  repertoire  given  that 
the  changes  do  not  affect  the  Test  Engine.  The  testing  will  typically  be  a subset 
of  the  original  Base  Registration  selected  to  test  those  aspects  of  the  MOT  that 
are  new  (e.g.,  documentation  and  ETC  analysis). 

Example: 

MOT(ATSx)/CLNP/LAN  ==>  MOT(ATSx+ATSy)/CLNP/l_AN 

B.5.5  Mutual  Recognition  Arrangements 

A Means  of  Testing  registered  by  another  authority  may  be  registered  with  the  Agent  of 
the  IGOSS  Panel*  if: 

(a)  A Memorandum  of  Understanding  exists  between  the  Agent  of  the  IGOSS 
Panel*  and  the  registering  authority,  recognizing  each  other's  assessment  and 
registration  procedures. 

(b)  Specific  application  is  made  to  the  Agent  of  the  IGOSS  Panel*  by  the  MOT 
supplier.  (Automatic  concatenation  of  registers  will  not  occur). 

(c)  The  candidate  MOT  for  IGOSS  registration  passes  the  Agent  of  the  IGOSS 
Panel*  static  evaluation. 

(d)  The  candidate  MOT  has  not  been  removed  from  the  Agent  of  the  IGOSS  Panel* 
register  as  a result  of  a successful  challenge  which  has  not  been  resolved  by  the 
primary  assessing  authority. 

f 

If  the  above  conditions  are  met,  the  candidate  MOT  will  be  registered  with  the  same 
expiration  date  as  for  the  primary  assessing  authority,  or  until  the  expiration  date  of  the 
associated  Abstract  Test  Suite,  whichever  is  earlier. 

If  an  MOT  which  is  registered  under  the  provisions  of  a Mutual  Recognition 
Arrangement  is  removed  from  the  primary  authority's  register  then  it  shall  be  removed 
from  the  IGOSS  register. 

The  Agent  of  the  IGOSS  Panel*  will  not  perform  on-line  assessments  of 
implementations  assessed  by  a mutually  recognized  authority.  The  Agent  of  the 
IGOSS  Panel*  will  review  the  application  for  completeness  and  plausibility  of 
information.  The  Agent  of  the  IGOSS  Panel  Is  the  final  authority  on  whether,  and  in 
what  circumstances,  registration  shall  occur. 

Applications  which  are  acceptable  to  the  Agent  of  the  IGOSS  Panel*  are  entered  in  the 
register  of  IGOSS  conformant  MOTs  under  the  same  categorization  as  the  primary 
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registration  (i.e.,  Registered  or  Provisionally  Registered)  and  with  the  same  constraints 
- Registration  is  based  on  the  expiration  of  the  Registered  Abstract  Test  Suite, 
Provisional  Registration  expires  at  the  same  time  as  that  of  the  primary  registration. 
The  MOT  supplier  will  be  notified  when  registration  is  approved.  The  register  will  be 
published  periodically,  and  will  be  available  on  request  from  the  Agent  of  the  IGOSS 
Panel*. 


B.6.  Dispute  Policy 

Any  Registered  implementation  may  be  challenged  by  any  interested  party  through  the 
Agent  of  the  IGOSS  Panel*.  The  challenger  will  pay  a challenge  fee  to  the  Agent  of  the 
IGOSS  Panel*  and  will  submit  a challenge  request  which: 

(a)  identifies  the  Implementation  being  challenged, 

(b)  names  one  or  more  tests  from  the  Executable  Test  Suite, 

(c)  describes  the  way  In  which  the  test(s)  are  Invalid. 

B.6.1  Challenges  to  Base  Implementations 

The  Agent  of  the  IGOSS  Panel*  will  send  the  challenge  to  the  MOT  supplier  asking  for 
comment.  If  the  MOT  supplier  does  not  adequately  address  the  problem  within  30  days 
then  the  MOT  is  statically  reassessed  according  to  the  criteria  in  B.3.2  of  this  Annex.  If 
the  MOT  no  longer  meets  the  criteria  for  the  category  in  which  It  Is  registered  then  the 
MOT'S  registration  is  revoked.  The  MOT  supplier  may  subsequently  demonstrate 
correct  operation  of  the  test  and  it  will  be  reinstated. 

B.6.2  Challenges  to  Derived  Implementations 

The  Agent  of  the  IGOSS  Panel*  will  send  the  challenge  to  the  MOT  supplier  asking  for 
comment.  If  within  30  days  the  supplier  is  unable  to  demonstrate  correct  operation  of 
the  test(s),  the  derived  implementation  will  be  marked  as  Provisionally  Registered  with 
an  expiration  date  a further  90  days  hence.  The  rules  for  Provisional  Registration  of  a 
base  Implementation  apply  thereafter. 

B.6.3  Challenges  to  'Mutual  Recognition  Arrangement'  implementations 

The  Agent  of  the  IGOSS  Panel*  will  send  the  challenge  to  the  primary  registering 
authority  asking  for  comment.  If  not  already  so  categorized  then  the  MOT  will  be 
marked  as  Provisionally  Registered  with  an  expiration  date  90  days  hence.  If  the 
primary  registering  authority  is  subsequently  satisfied  that  the  test(s)  operate  correctly 
then  the  former  categorization  is  reinstated. 
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B.6.4  Abstract  Test  Suite  Problem  Resolution 

In  the  long  term  it  is  hoped  that  Abstract  Test  Suites  and  related  test  case  Interpretation 
issues  will  be  authorized  by  the  relevant  ISO  working  groups.  However  a more 
Immediately  responsive  forum,  technically  grounded  and  sufficiently  democratic,  is  the 
OSE  Implementors'  Workshop,  through  the  Conformance  Testing  SIG  and  the 
appropriate  technical  SIGs.  The  GOSIP  Testing  Program  will  accept  rulings  on  test 
case  Interpretation  from  this  forum. 

B.6.5  Reference  Implementation  Problem  Resolution 

For  each  IGOSS  protocol  the  Agent  of  the  IGOSS  Panel*  maintains  a Reference 
Implementation,  or  identifies  a 'known'  implementation.  In  either  case  it  is  unlikely  that 
this  implementation  will  be  completely  bug-free.  Fortunately,  for  the  purpose  of  MOT 
assessment  it  Is  not  either  necessary,  or  even  desirable,  to  maintain  a perfect 
implementation.  It  is  however,  necessary  for  the  Agent  of  the  IGOSS  Panel*  to 
maintain  a catalogue  of  the  known  problems  with  the  implementation.  It  is  appropriate 
that  during  assessment,  an  MOT  is  able  to  discover  these  problems  and  mark  the 
appropriate  tests  with  a 'Fall'  or  'Inconclusive'  Verdict. 

Consequently,  any  new  problems  found  with  the  known  implementatioa  will,  after 
corroboration,  be  added  to  the  catalogue  of  errors.  Any  fixes  will  be  deleted  from  the 
catalogue. 
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ANNEX  C:  HOW  TO  ACCESS  THE  REGISTERS 
C.1  On-Line  Registers 

The  United  States  Government  Open  Systems  Interconnection  Profile  (GOSIP)  Testing 
Program  was  defined  to  permit  Federal  Agencies  to  substantiate  claims  of  GOSIP 
compliance.  As  a result  of  this  program,  Federal  Agencies  can  ensure  that  GOSIP 
compliant  products  are  effectively  tested  for  conformance  to  the  Open  Systems 
Interconnection  (OSI)  standards  and  for  interoperability  with  other  OSI 
Implementations.  The  IGOSS  Testing  program  has  been  built  up  from  the  GOSIP 
Testing  program. 

This  document  produced  by  the  National  Institute  of  Standards  and  Technology  (NIST) 
entitled  "IGOSS  Testing  Framework"  establishes  the  framework  by  which  IGOSS 
Acquisition  Authorities  can  ensure  that  procured  IGOSS  products  are  compliant  with  the 
IGOSS  specifications.  The  vehicle  by  which  this  determination  can  be  made  is  a set  of 
registers.  These  include  registers  for  Test  Suites,  Test  Systems  (Means  of  Testing), 
Conformance  Testing  Laboratories,  and  Interoperability  Testing  Services. 

The  U.S.  GOSIP/IGOSS  Register  Database  is  an  online  database  facility  developed  by 
NIST  and  JITC.  It  provides  up-to-date  reference  information  for  the  following  list  of 
registers: 

1.  IGOSS  PICS  Proforma 

2.  IGOSS  Abstract  Test  Suites  (ATS). 

3.  Assessed  Means  of  Testing  (MOT). 

4.  NVLAP  or  SCC  Accreditated  Test  Laboratories. 

5.  Conformance  Tested  IGOSS  Products. 

6.  Interoperability  Test  Suites  (ITS)  for  OSI  Products. 

7.  Reference  Entities  for  Means  of  Testing  Assessment(s). 

8.  Interoperability  Test  and  Registration  Services. 

9.  Approved  IGOSS  MOT  Validation  Laboratories. 

To  access  the  GOSIP/IGOSS  registers,  follow  these  instructions: 

C.1.2  To  reach  the  Database 

Dial  in  on  a 1200/2400  baud  analog  modem  (Bit/Char:  8,  Stop  Bits:  1,  Parity:  None)  to 
(602)  538-5233.  There  are  actually  four  lines  on  rotary  to  ensure  access.  Once  you 

are  connected,  you  will  see  the  login  prompt.  Respond  with  the  user  name  "jitcl"  In  the 
lower  case  only.  No  password  is  needed.  You  will  be  asked  for  your  name,  and  then 
asked  to  designate  the  terminal  emulation  you  use.  You  will  then  sequence  through  a 
series  of  menu  screens  that  allow  you  to  access  and  read  GOSIP/IGOSS  register  data 
and  generate  ASCII  text  report  files  to  download  to  your  computer. 
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JITC  has  tested  dial-in  access  and  file  downloading  using  Procomm  Plus  and  Kermit  on 
a PC. 

You  can  also  access  the  Database  through  the  Internet.  The  host  name  is  huachuca- 
jitcosi.army.mil  and  the  Internet  protocol  address  is  138.27.7.2.  Two  separate  logins 
are  required,  both  with  user  name  "jitcl"  and  no  password.  The  first  login  activates  a 
shell  script  that  immediately  dials  out  to  connect  to  the  Database.  If  the  outgoing  line  is 
busy,  the  user  will  be  notified  and  then  automatically  logged  out. 

All  data  access  functions  are  supported  over  the  Internet,  but  file  downloading  Is  not 
supported. 

C.1.2  For  server  access  to  downloadable  files: 

The  GOSIP/IGOSS  register  files  for  all  GOSIP/IGOSS  registers,  along  with  the  product 
and  MOT  registration  application  forms,  are  available  on  a host  computer  which 
supports  faster  downloading.  The  files  are  available  in  WordPerfect  Version  5.1  and 
ASCII  text  formats. 

You  can  access  the  downloadable  flies  on  this  host  through  dial-ln  modem  or  through 
the  Internet. 

You  can  dial  in  on  a 2400  baud  analog  modem  (8-1-N-2400)  to  (602)  538-5004/4323, 
or  on  a 9600  baud  digital  modem  (8-1-N-9600)  to  (602)  538-5034/5036.  Once  you 
have  accessed  the  host,  hit  "Enter."  This  will  bring  up  an  alert  screen,  and  ask  for  your 
login  name.  At  the  login  prompt,  type  "jitcl"  in  lower  case  or  upper  case.  Again,  no 
password  is  needed.  A menu  will  be  displayed.  You  may  read  a page  of  instructions, 
review  register  files,  and  download  register  files. 

If  you  decide  to  download  files,  you  have  to  access  Kermit.  The  Instructions  will  tell  you 

to  select  the  download  choice,  and  at  the  prompt  to  hit  "PgDn."  That  gives  you  several 
selections,  and  you  should  choose  Kermit. 

To  download  GOSIP/IGOSS  register  files  from  the  host  over  the  Internet,  ftp  to  Internet 
Protocol  address  138.27.8.2.  Login  as  "anonymous"  and  give  you  user  name  as  the 
password.  Change  directories  to  the  GOSIP  directory  (e.g.,  cd  gosip).  Then  list  the 
files  (e.g.,  Is).  For  the  WordPerfect  files,  you  need  to  type  "binary."  Then  type  in 
"quote  site  io_mode."  Then  you  can  "get"  the  files.  For  ASCII  files,  you  can  omit  the 
"binary"  and  "quote  site  io_mode"  commands. 

In  order  to  improve  the  service,  JITC  has  established  an  additional  host  for  Internet 
downloading  of  GOSIP/IGOSS  register  files.  For  the  new  host,  the  procedure  is  to  ftp 
to  huachuca-jitcosi.army.mil  (138.27.7.2),  login  as  "anonymous"  and  a text  file  of 
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instructions  and  subdirectories  for  GOSIP/IGOSS  register  files  and  registration  forms. 
Both  ASCII  text  files  and  WordPerfect  files  are  included.  Use  "binary"  when  transferring 
WordPerfect  files. 

JITC  has  established  a new  telephone  number  for  GOSIP/IGOSS  registration  support, 
including  assistance  in  accessing  the  register.  The  number  to  call  Is  (602)  538-4351. 

C.2  Validated  Products  List 

The  Validated  Products  List  (VPL)  is  published  quarterly  and  currently  distributed  by  the 
National  Technical  Information  Service  (NTIS).  For  order  please  contact: 

National  Technical  Information  Service 
U.S.  Department  of  Commerce 
5285  Port  Royal  Road 
Springfield,  VA  22161 

for  subscriptions  call:  (703)  487-4630 

for  individual  copies:  (703)  487-4650 

The  entries  In  the  Validated  Products  List  may  be  accessed  on  the  Internet  using  the 
following  Instructions: 

type:  ftp  speckle.ncsl.nlst.gov 

(internet  address  129.6.59.2) 

Login  as  user  ftp 

type  your  e-mail  address  as  the  password 
type:  cd  vpl  or  cd  pub/vpl 
type:  binary 

The  directory  contains  the  full  VPL,  type  Is  for  complete  list. 


Version  1.0 


June  1,  1994 


Page  87 


Annex  C:  How  to  Access  the  IGOSS  Registers 


This  Page  is  intentionally  blank 


Version  1 .0 


June  1,  1994 


Annex  D:  Advice  to  Procurement  Authority 


ANNEX  D:  ADVICE  TO  PROCUREMENT  AUTHORITY 


D.1  Introduction  and  Statement  of  the  Problem 

The  IGOSS  Testing  Program  has  been  set  up  from  the  U.S.  GOSIP  Testing  Program. 
As  such  Acquisition  Authorities  inherit  the  same  type  of  procurement  strategy.  This 
Annex  refers  to  the  U.S.  GOSIP  Testing  Program  as  an  example. 

Following  the  publication  of  FIPS  146  [NIST  17],  it  became  mandatory  for  Federal 
Agencies  to  procure  networking  products  based  on  the  Open  Systems  Interconnection 
specifications  as  from  August  1990.  With  the  publication  of  FIPS  146-1  [NIST  18],  the 
required  functionality  were  expanded,  from  FTAM  and  X.400  over  LAN  and  WAN 
technologies,  to  embrace  Virtual  Terminal  applications  and  ISDN.  With  these  complex 
and  expensive  technologies,  it  Is  not  enough  for  vendors  to  make  unsubstantiated 
claims  of  compliance.  It  Is  necessary  to  have  a demonstration  of  compliance. 

This  Annex  discusses  how  the  IGOSS  Testing  Policy  can  be  used  to  assist  in  Federal 
procurements,  and  what  extra  steps  an  Acquisition  Authority  must  take  to  secure  quality 
products. 

D.2  The  IGOSS  Registers 

The  visible  result  of  the  IGOSS  Testing  Policy  is  a database  of  Registers  listing  the 
various  elements  of  the  program.  The  complete  list  of  Registers  is: 

1)  IGOSS  PICS  Proformae 

2)  IGOSS  Abstract  Test  Suites 

3)  Interoperability  Test  Suites 

4)  Validated  Means  of  Testing 

5)  NVLAP  or  SCC  Accredited  Conformance  Testing  Laboratories 

6)  IGOSS  ReferencerSample"  Implementations 

7)  Conformance  Tested  IGOSS  Products 

8)  Interoperability  Testing  Services 

9)  MOT  Validation  Laboratories 

The  single  register  of  most  use  for  procurement  purposes  is  the  Register  of 
Conformance  Tested  IGOSS  Products.  Products  successfully  tested  in  a NVLAP  or 
SCC  Accredited  Laboratory  are  entered  onto  this  Register.  The  next  most  useful 
register  is  the  Interoperability  Testing  Services  Register,  which  Is  a list  of  services 
approved  to  conduct  and  advertise  pairwise  interoperability  testing  between  IGOSS 
products. 
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D.2.1  The  Register  of  Conformance  Tested  IGOSS  Products 

IGOSS  products  which  have  been  successfully  tested  by  an  accredited  test  laboratory 
using  a registered  Means  of  Testing,  including  registered  Abstract  Test  Suites,  may  be 
added  to  the  register  of  Conformance  Tested  IGOSS  Products.  The  goal  of 
conformance  testing  is  to  verify  product/protocol  compliance  to  the  relevant  standard. 

The  register  contains  a list  of  products  that  have  successfully  passed  conformance 
testing  (i.e.,  not  demonstrated  any  instance  of  non-compliance  during  the  test 

campaign).  Conformance  testing  is  not  a 100%  warranty  that  the  product  you  are 
purchasing  Is  compliant.  It  does  indicate  that  a minimum  amount  of  testing  using 

globally  accepted  test  specification  has  been  applied  successfully  to  the  products 
registered. 

Testing  of  OSl  End  Systems  has  been  modularized  to  separate  supporting  network 
connectivity,  intermediate  layers,  and  applications,  and  the  structure  of  the  IGOSS 
Products  Register  reflects  these  separations.  There  are  separate  categories  such  as: 

WAN  Products  (X.25,  ISDN,  Frame-Relay) 

LAN  Products  (802.3,  802.4,  802.5) 

CLNP  Products  (Intermediate  Systems) 

Transport  Products  (TP2,  TPO,  and  TP4/CLNP  End  Systems) 

Session  Products 

X.400  Products  (Message  Handling  Systems)  1984  and  1988 
FTAM  Products  (File  Transfer  Applications) 

X.500  Products  (Directory  Service  Application) 

MMS  Products  (Manufacturing  Message  Specification  Application) 

Network  Management  Products 

For  End  Systems  procurement,  your  concern  is  with  the  X.400,  FTAM,  and  Virtual 
Terminal  categories.  The  Session,  Transport,  LAN  and  WAN  categories  provide 
supporting  connectivity  for  these  applications.  For  Intermediate  Systems/Gateway 
procurements,  your  concern  is  with  the  CLNP  products  category.  LAN  and  WAN 
categories  also  provide  supporting  connectivity  to  Intermediate  Systems  (or  "Routers"). 
Entries  in  each  category  are  as  follows: 

The  name  of  the  Register  category,  e.g.  X.400  Products. 

Supplier  Name  and  Address. 

Contact  name,  with  Phone  and  Fax  Numbers. 

GOSIP  Product  Name,  Release  and  Date,  e.g.  ACME  Software  X.400 
Messager,  Version  27.92,  October  1992. 
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Hardware  and  Operating  System  Platform,  e.g.  ACME  Hardware  2900  Series, 
ACMIX  Version  14.7. 

Base/Derived,  i.e.  whether  the  product  was  fully  tested  on  this  platform  (Base), 
or  ported  from  another  HAA/  0/S  platform  (Derived). 

Connectivity,  i.e.  in  the  case  of  an  X.400  application,  the  associated  registered 
Session  platform. 

Protocols  and  Profiles,  e.g.  CCITT X.400  1984  Series,  PI,  P2  and  RTS. 

Date  Registered. 

Type  of  Registration,  I.e.  Provisional  or  Full.  Depends  on  the  state  of  maturity  of 
the  Abstract  Test  Suite  and  Means  of  Testing.  Full  Registration  only  occurs 
after  the  Abstract  Test  Suite  has  reached  International  Standard,  and  test  tool 
coverage  is  at  the  95%+  level.  For  Provisional  Registration,  coverage  is  at  60%+ 
or  80%+.  In  this  context,  coverage  is  only  a measure  of  the  minimum 
requirements  for  test  tool  vis-a-vis  the  test  suite  (I.e.,  if  a test  tool  implements 
properly  69  test  cases  out  of  100  test  cases  in  the  registered  abstract  test  suite, 
the  test  tool  is  said  to  cover  at  69%).  There  is  no  clear  relationship  between  this 
measure  and  how  much  of  the  protocol  is  tested. 

Conformance  Lab.  Used,  e.g.  the  ACME  Conformance  Testing  Center  (Should 
be  on  the  list  of  NVLAP  or  SCC  Accredited  Labs). 

Some  products  In  the  register  have  been  grandfathered  from  one  GOSIP/IGOSS 

version  to  another.  Grandfatherino  is  the  act  of  granting  GOSIP/IGOSS  Version  N+1 
conformance  testing  registration  to  a GOSIP/IGOSS  Version  N registered  product 
without  requiring  conformance  testing.  Such  products  are  clearly  identified  in  the 
register. 

For  each  product  registered  the  vendor/bidder  has  the  ability  to  provide  you  with 
complementary  information  about  the  product  functionality.  A set  a documents  was 
employed  during  the  test  campaigns  and  describe  completely  each  protocol  layer 
capability.  These  documents  are  the  Protocol  Implementation  Conformance 
Statements  (PICS)  and  Protocol  Implementation  eXtra  Information  for  Testing  (PIXIT). 
The  PICS  is  a statement  made  by  the  supplier  of  an  OSI  Implementation,  or  system, 
stating  which  capabilities  and  options  have  been  implemented,  for  a given  OSI  protocol. 
The  PIXIT  is  a statement  made  by  a supplier  or  Implementor  of  an  lUT  which  contains 
or  references  all  of  the  Information  (in  addition  to  that  given  in  the  PICS)  related  to  the 
lUT  and  its  testing  environment,  which  will  enable  the  test  laboratory  to  run  an 
appropriate  test  suite  against  the  lUT.  These  documents  can  be  used  by  technical 

experts  to  assess  how  the  products  match  the  actual  requirements.  These  documents 
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can  be  easily  used  to  verify  that  the  product  proposed  matched  the  actual  product 

tested  and  registered. 

D.2.2  The  Register  of  Interoperability  Testing  Services 

This  register  contains  the  addresses  and  point  of  contact  of  the  registered 
interoperability  testing  services: 

Each  of  these  organizations  maintains  a register  of  interoperability  tested  products, 
which  have  been  tested  against  the  Interoperability  Test  Suites. 

Each  organization  uses  different  procedures.  You  are  advised  to  get  a copy  of  the 
relevant  documentation  and  assess  whether  the  organization  matches  you  quality 
requirements.  For  each  interoperability  test  campaign,  each  organization  should: 

1)  Conduct  a static  analysis  phase  which  involves  the  selection  of  a common 
subset  of  the  IGOSS  tests  including  all  of  the  mandatory  tests  and  all  the 
relevant  optional  tests  resulting  from  the  implementation  of  the  same  optional 
functionality; 

2)  Conduct  a dynamic  analysis  phase  in  which  both  IGOSS  product  suppliers  are  in 
agreement  concerning  the  outcome  of  each  test; 

3)  Issue  a test  report  which  identifies  each  IGOSS  product  supplier,  describes  the 
product  Including  the  supporting  stack  of  protocols,  and  provides  the  list  of  the 
tests  selected  and  executed  with  a verdict  for  each  test.  The  verdict  may  be 
'pass'  or  'fail'.  Any  fail  verdict  must  be  accompanied  by  an  explanation  outlining 
the  cause  of  failure.  Any  non  selected  test  case  must  be  accompanied  by  an 
explanation  outlining  the  rational  for  de-selection. 

You  may  require  a copy  of  the  Interoperability  testing  test  report  and  verify  that  the 
product  has  successfully  demonstrated  all  the  claimed  functionality. 

D.3.  Procurement  Strategy 
The  following  text  Is  recommended: 

"Any  computer  networks  and  communications  equipment,  systems  and  services  offered 
in  response  to  the  specific  functional  requirements  in  this  solicitation  must  provide  the 
associated  protocols  specified  In  the  standards  referenced  by  IGOSS  or 
Industry/Government  Open  Systems  Specification." 

However  there  are  some  Important  things  to  note: 
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1)  The  IGOSS  Testing  Program  specifies  communications  technology  only,  other 
Information  Processing  concerns  are  outside  its  scope. 

2)  We  recognize  that  many  procurements  are  for  larger  scale,  general  purpose 
ADP  (Automated  Data  Processing)  acquisitions,  involving  a mixture  of  different 
specification.  There  are  very  many  ways  of  combining  these  specification. 
There  are  very  many  ways  of  combining  the  protocol  profiles  within  the  IGOSS. 
Business  Unit  functional  requirements  are  what  determine  the  mix. 

The  general  wording,  above,  explicitly  allows  the  possibility  of  acquiring  individual  slices 
through  the  IGOSS  profile.  However,  this  requires  that  the  solicitation  be  augmented 
with  a detailed  schedule  of  actual  IGOSS  applications  and  connectivity  which  are 
sought. 

D.3.1  The  Schedule 

For  Instance  if  the  Acquisition  Authority  requirement  is  for  Electronic  Message  Handling 
Service,  then  the  X.400  application  is  what  you  need  to  acquire.  The  need  may  be  for 
a service  attached  to  a public  network,  or  for  a private  messaging  service.  The 
alternative  full  stacks  for  these  options  are: 

Public  Messaging  Service  attachment: 

X.400/Session/Transport  Class  0/X.25 

Private  Messaging  Service: 

X.400/Session/Transport  Class  4/CLNP/LAN  or  WAN. 

Where  WAN=X.25;  LAN=802.3,  802.4,  or  802.5. 

More  likely,  you  will  need  to  acquire  both  TPO  and  TP4  solutions  to  encompass 
U\N/WAN  inter  networking  situations.  If  your  solicitation  includes  at  least  the 
Applications  and  the  LAN  and/or  WAN  networks,  then  this  can  be  used  in  conjunction 
with  the  appropriate  general  statement  above,  to  define  the  relevant  portions  of  IGOSS 
to  be  provided. 

D.3.2  Specification  of  Options 

In  some  cases  the  base  (ISO/CCITT)  standards  provide  optional  functions  which  are 
not  resolved  in  the  Stable  Implementation  Agreements  [NIST  1]  or  In  FIPS  146-2.  Any 
such  options  required  by  an  Acquisition  Authority  must  be  explicitly  called  out  and 
verified  with  a supplier,  who  may  or  may  not  have  chosen  to  implement  such  options. 
[See  also  D.4.5]. 

D.3.3  Assurance  of  Troubleshooting/Interoperability  Support 

The  question  of  Interoperability  Assurance  prior  to  acquisition  and  installation  is 
covered  in  section  D.4.4  below.  However  it  should  be  noted  that  even  a combination  of 
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conformance  and  interoperability  testing  cannot  give  guarantees  of  complete 
interworking.  Inevitably,  problems  will  develop  after  Installation,  both  within  the  scope 
of  the  acquisition  and  with  other  IGOSS  systems.  For  this  reason  It  Is  Important  to 
secure  cooperative  maintenance  arrangements  with  IGOSS  suppliers,  such  that  each 

supplier  associated  with  an  Interoperability  problem  is  required  to  participate  in  its 
resolution.  This  can  easily  be  arranged  by  contract,  within  the  scope  of  an  acquisition  - 
but  there  are  several  possibilities: 

1)  (Intra  Business  Unit,  Intra  Procurement)  - Interoperability  problem  between 
suppliers  under  the  scope  of  a single  acquisition:  as  above. 

2)  (Intra  Business  Unit)  - Interoperability  problem  between  suppliers  of  any  two 
systems  within  an  Business  Unit:  therefore  any  supplier  of  IGOSS  products  to 
an  Business  Unit,  irrespective  of  when  the  acquisltion(s)  occurred,  must  commit 
to  a cooperative  maintenance  arrangement. 

3)  (Inter  Business  Unit)  - Interoperability  problem  between  suppliers  of  systems  to 
different  Business  Units:  therefore  any  pair  of  suppliers  of  IGOSS  products  to 
Business  Units  Involved  in  the  problem,  must  be  prepared  to  assist  in  problem 
resolution.  For  this  reason  every  acquisition  must  require  participation  in 
problem  resolution  on  the  part  of  any  involved  supplier  who  provides  a product  to 
any  Business  Unit  at  any  time. 

4)  (Business  Unit  - Rest  of  the  World)  - Interoperability  problem  between  suppliers 
of  systems  to  an  Business  Unit  and  any  other  organization:  recourse  to  the 
other  supplier  is  probably  limited,  except  insofar  as  that  supplier  may  have 
entries  on  IGOSS  conformance  or  interoperability  registers  which  could  be 
jeopardized  by  non-cooperation. 

In  summary,  the  best  assurance  of  government-wide  interoperability  can  be  gained  by 

every  Business  Unit  requiring  that  all  IGOSS  suppliers  commit  to  participating  in  the 
resolution  of  any  Interoperability  problem  in  which  their  system  is  involved.  This  can  be 
greatly  facilitated  by  the  encouragement  of  all  IGOSS  suppliers  to  participate  in  pre- 
procurement Interoperability  Testing;  and  in  addition,  that  post-procurement 
Interoperability  Analysis  services  be  widely  recognized  and  used. 

D.4  Acquisition  Testing  Strategy 
D.4.1  Introductory  Discussion 

Systems  offered  in  response  to  a solicitation  which  includes  IGOSS  functionality  must 
be  demonstrated,  or  have  been  demonstrated,  to  be  IGOSS  compliant.  The  only  way 
to  make  this  determination  is  by  successful  conformance  testing  in  a NVLAP  or  SCO 
accredited  laboratory,  leading  to  entry  on  the  Conformance  Tested  IGOSS  Products 
Register.  This  requirement  should  be  applied  on  a protocol  by  protocol  basis.  There 
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are  some  protocols  for  which  conformance  testing  infrastructure  (Abstract  Test  Suites 
and  Means  of  Testing)  does  not  yet  exist  - in  particular,  the  Virtual  Terminal  application. 
A list  of  validated  IGOSS  products  can  be  found  in  the  NIST  Validated  Products  List  [5] 
produced  by  the  NIST  Computer  Systems  Laboratory  and  published  quarterly 
(December,  March,  June,  September).  However  the  most  current  list  of  products  may 
be  found  In  the  GOSIP/IGOSS  Database  Register.  This  register  contains  only  products 
that  have  been  fully  validated.  It  does  not  contain  products  available  but  not  tested,  nor 
products  under  test. 

In  certain  technical  areas,  the  GOSIP/IGOSS  Products  Register,  as  it  stands  right  now, 
may  not  be  well  populated  with  tested  applications.  In  order  to  achieve  an  acceptable 
mix  of  competitive  bids  an  Acquisition  Authority  may  be  compelled  to  solicit  and  review 
bids  for  products  which  are  not  yet  tested. 

Once  a requirement  for  functionality  exists,  one  or  more  of  the  following  problem 
conditions  may  exist:  1)  no  products,  or  only  a very  limited  number  of  products  are 
available;  2)  conformance  tests  are  not  available;  3)  a very  limited  number  or  no 
products  have  passed  conformance  testing.  Test  availability  is  defined  as 
conformance  tests  being  available  within  at  least  one  third  party  Accredited 
Conformance  Test  Laboratory. 

The  following  is  assumed:  if  products  are  available  and  a test  is  available,  then 
vendors  are  at  least  attempting  to  pass  the  conformance  test;  if  no  or  only  a very 
limited  number  of  validated  products  are  available,  then  the  Acquisition  Authority  will 
want  to  give  vendors  time  to  successfully  validate  each  product. 

D.4.2  Validation  Options 

The  following  definitions  apply:  PV  is  Prior  Validation,  LIT  Is  Linder  Test  (formerly  - 
Prior  Validation  Testing),  DV  is  Delayed  Validation,  DD  is  Delayed  Delivery. 


Products  Available 

Y 

Y 

Y 

Y 

V 

V 

V 

N 

N 

Test  Available 

Y 

Y 

Y 

N 

Y 

Y 

N 

Y 

N 

PrbYf  ucts  Va  1 id  ated 

Y 

V 

N 

V 

N 

Paragraph 

PV 

UT 

UT 

DV 

UT 

UT 

DV 

DD 

DD 

Wh'eTe  Y^Ye^syVWery  limited  Number;  ^ 


^No 


Prior  Validation  (PV) 

"The  offeror  shall  certify  in  the  offer  that  Implementations  of  the  appropriate  protocol, 
including  applicable  options,  offered  in  response  to  this  document  have  been  previously 
validated". 
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Note  that  Prior  Validation  is  demonstrated  by  a product's  inclusion  in  the  IGOSS 

Product  Register.  There  are  currently  NO  certificates  issued,  nor  are  there  Validation 
Summary  Reports  (VSR)  available  for  IGOSS  Products. 

Under  Test  (UT) 

"The  offeror  shall  agree  to  correct  all  Implementation  non  conformance  from  the 
applicable  protocol.  All  areas  of  non  conformance  must  be  corrected  within  [a  specific 
period  of  time  i.e.  3 to  6 months]  from  the  date  of  contract  award.  If  an  interpretation  of 
IGOSS  is  required  that  will  invoke  the  procedures  set  forth  such  these  in  FIPS  29-2, 
such  a request  for  interpretation  shall  be  made  within  30  calendar  days  after  contract 
award.  Any  corrections  that  are  required  as  a result  of  decisions  made  under  the 
relevant  procedures  shall  be  completed  within  (a  specific  time  i.e.  3-6  months  of  the 
date  of  formal  notification  to  the  contractor  of  the  approval  of  the  interpretation.  Proof 
of  correction  in  either  case  shall  be  Inclusion  on  the  IGOSS  Products  Register.  Failure 
to  make  the  required  corrections  within  the  time  limits  set  forth  above  shall  be  deemed 
a failure  to  deliver  required  software.  The  liquidated  damages  as  specified  for  failure  to 
deliver  the  operating  system  or  other  software  shall  apply." 

Delayed  Validation  (DV) 

"The  offeror  shall  certify  In  the  offer  that  the  implementation  of  the  protocol.  Including 
applicable  options  offered  in  response  to  this  document  will  be  submitted  for  validation 
and  validated  within  6 months  after  conformance  testing  services  are  available.  Unless 
specified  elsewhere,  proof  of  submission  for  validation  shall  be  in  the  form  of  a letter 
scheduling  the  test  campaign.  Issued  by  an  Accredited  Conformance  Testing 
Laboratory.  Once  the  product  is  Under  Test  the  conditions  of  that  option  apply." 

You  can  estimate  when  validation  tests  will  become  available  by  reviewing  the  Register 
of  Abstract  Test  Suites  and  the  Register  of  Validated  Means  of  Testing. 

Delayed  Delivery  (DD) 

If  none  or  a very  limited  number  of  products  are  available  then  the  Acquisition  Authority 
should  consider  postponing  the  delivery  of  the  requirement  until  either  the  conditions  of 
"Under  Test"  or  "Delayed  Validation"  exist. 

D.4.3  Adding  IGOSS  Capability  to  Existing  Installations 

All  of  the  foregoing  applies  to  the  procurement  and  testing  of  combinations  of  hardware 
and  software,  representing  new  acquisitions.  The  case  of  adding  IGOSS  capability  to 
existing  Installations  Is  broadly  the  same,  except  in  this  case  a sole-source  justification 
may  also  be  required.  Other  than  this,  the  choice  of  Prior  Validation,  Under  test. 
Delayed  Validation  and  Delayed  Delivery  is  as  above. 

D.4.4  Interoperability  Testing  as  Extra  Insurance  for  Product  Acceptability 
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Conformance  testing  is  good  because  it  helps  to  demonstrate  a product's  conformance 
to  the  published  standards;  however,  on  its  own  it  does  not  provide  a guarantee  of 
interoperability  between  conformance  tested  products.  Interoperability  testing  is  good 
because  it  provides  a method  for  IGOSS  product  implementations  to  achieve 
interworking;  however  on  its  own,  without  conformance  testing,  the  patches  applied  to 
enable  product  Interworking  may  cause  drift  away  from  the  standards,  thereby  making  it 
difficult  for  subsequent  suppliers  to  attach  their  conformant  Implementations  to  the 
installed  base.  Therefore  the  optimal  long  term  goal  is  to  achieve  conformant 
interoperability.  Although  Interoperability  testing  is  not  mandated  In  the  IGOSS  Testing 
Program,  an  Acquisition  Authority  should  require  it. 

D.4.5  Extra  Testing  Possibilities 

D.4.5.1  Security  Features 

At  present  there  is  no  testing  program  in  place  for  any  of  the  security  options  within 
IGOSS.  An  Acquisition  Authority  wishing  to  acquire  these  options  must  be  prepared  to 
specify  its  own  conformance  and  acceptance  tests. 

D.4.5.2  Performance  Requirements 

At  this  early  stage  of  the  deployment  of  IGOSS,  there  Is  Insufficient  Installed  base  to 
develop  meaningful  performance  statistics.  Any  Acquisition  Authority  wishing  to  levy 
performance  requirements  on  IGOSS  procurements  must  first  develop  the  performance 
criteria  and  then  arrange  to  have  all  applicable  products  exercised  in  order  to  apply  the 
criteria.  It  is  unlikely  that  very  many  implementations  will  meet  stringent  performance 
criteria  today.  Therefore,  the  IGOSS  Testing  Program  recommends  that  an  Acquisition 
Authority  develop  its  own  performance  criteria,  or  encourage  the  development  of 
generic  performance  criteria,  and  gain  experience  in  measuring  Implementations  for 
later  use  In  requirements  setting. 

D.4.6  A Note  on  Diagnostic  Equipment 

In  order  to  troubleshoot  IGOSS  problems  It  may  be  necessary  for  Acquisition 
Authorities  to  acquire  special-purpose  diagnostic  equipment.  Such  analyzers  have  long 
existed  in  the  form  of  line  monitors,  LAN  monitors  and  X.25  packet  decoders.  However, 
one  can  expect  the  burgeoning  OSI  market  to  precipitate  the  need  for,  and  supply  of, 
multi-layer  OSI  protocol  decoders,  spanning  one  or  more  layers  of  protocol  from  l_AN  to 
Application.  There  is  today  no  requirement  In  the  IGOSS  Testing  Program  to  test  and 
register  this  equipment.  When  procuring  such  an  analyzer  an  Acquisition  Authority 
should  specify  tests  of  its  diagnostic  capability  for  each  protocol  for  which  support  Is 
claimed.  Presenting  the  results  of  a decoded  Interoperability  test  may  be  one  way  to 
do  this.  Presenting  the  results  of  a decoded  Conformance  Test  is  probably  better, 
since  all  Protocol  Data  Unit  fields  are  spelled  out  in  each  Conformance  Test. 
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D.4.7  Holes  in  the  Testing  Infrastructure 

There  are  some  cases  where  an  Abstract  Test  Suite  and/or  a Means  of  Testing  do  not 
exist  for  particular  IGOSS  protocols.  The  three  Validation  options  given  above  are 
inadequate  to  deal  with  these  cases.  You  can  determine  what  these  cases  are  by 
examining  the  Register  of  Abstract  Test  Suites  and  the  Register  of  Validated  Means  of 
Testing.  It  is  possible  that,  for  instance,  Virtual  Terminal  products  may  become 
available  and  the  means  for  validating  them  be  indefinitely  delayed.  Therefore  even 
employment  of  "Delayed  Validation"  wording  will  not  yield  Validated  Products  by  the 
time  of  delivery  to  the  Acquisition  Authority.  If  you  have  available  the  qualified  technical 
resource,  you  may  develop  a specific  acceptance  testing  program. 

D.5  Sample  Text 

This  section  is  for  information  only  and  provides  guidance  on  how  to  describe  IGOSS 
testing  requirements. 

[PROCUREMENT-NAME]  - Statement  of  Work: 

General  Requirements:  "...  requires  that  all  computer  network  and  data 
communications  systems  and  services  procured  comply  with  the  requirements  of  the 
IGOSS  [REF]  and  the  IGOSS  Testing  Framework  [REF]." 

1.  "The  technical  objective  standard  for  PROCUREMENT-NAME]  is  IGOSS 
Version  1.0.  Because  Abstract  Test  Suites  (ATS)  may  not  be  available  for 
IGOSS  Version  1.0  testing  In  time  to  meet  the  [PROCUREMENT-NAME] 
acquisition  schedule,  the  Contractor  shall  Implement  the  network  using 
equipment  that  appears  on  the  NIST  Register  of  GOSIP  compliant  products, 
tested  at  least  to  GOSIP  Version  2.0.  At  such  time  IGOSS  1.0  ATS  are 
available,  the  Contractor  shall  retest  applicable  products  for  compliance  with 
IGOSS  1.0.  Should  the  equipment  fail  IGOSS  1.0  testing,  the  Contractor  shall, 
at  his  cost,  take  whatever  remedial  actions  are  necessary  to  bring  the  equipment 
to  IGOSS  1.0  standards  and  have  it  retested.  Alternatively,  the  Contractor  may 
elect  to  replace  that  equipment,  at  his  cost,  with  another  vendor's  equipment  that 
does  meet  IGOSS  1.0  standards." 

2.  "The  Contractor  may  propose  use  of  untested  products  having  GOSIP  2.0 
functionality  to  meet  a specified  [PROCUREMENT-NAME]  requirement  provided 
testing  is  completed  by  the  time  of  submission  of  Best  and  Final  Offers  (BAFO). 
Should  testing  not  be  completed  by  the  time  of  submission  of  BAFO,  the 
Contractor  shall  offer  a substitute  piece  of  equipment  which  has  been  GOSIP  2.0 
registered." 

3.  "In  the  event  there  are  no  IGOSS  certified  products  available  for  substitution,  or 
there  is  no  applicable  ATS  for  the  item,  or  If  there  are  no  IGOSS  compliant 
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products  available  in  industry  to  meet  the  specified  requirement,  then  and  only 
then  may  a non-IGOSS  compliant  item  be  offered.  If  this  occurs,  the  item  shall 
be  identified  as  a IGOSS  compliance  deficiency  in  the  migration  plan  required  by 
[MIGRATION-TEXT-REF]." 

4.  "Should  a registered  IGOSS  product  suitable  for  an  application  identified  as  a 
IGOSS  deficiency  In  the  IGOSS  migration  plan  become  available  in  the  interim, 
the  Contractor  shall  have  180  days  to  either  replace  his  non-equipment  with  the 
available  certified  equipment,  or  make  his  equipment  compliant  and  complete 
certification  testing.  The  Acquisition  Authority  will  discontinue  paying  recurring 
charges  on  any  non-certified  equipment  being  used  In  the  network  180  days 
after  certified  equipment  that  meets  the  functional  requirements  becomes 
available  on  the  market." 

Definition: 

IGOSS  Compliant:  "Is  defined  as  the  Contractor  Implementing  IGOSS  protocols  in 
accordance  with  IGOSS  and  providing  equipment  for  use  in  [PROCUREMENT-NAME] 
with  IGOSS  functionality  that  adheres  to  the  provisions  of  the  IGOSS  Testing  and 
Registration  Program  as  described  In  the  IGOSS  Testing  Framework  Technical  Report 
[REF]  and  that  appears  on  the  NIST  Register  of  Conformance  Tested  GOSIP/IGOSS 
Products." 

Note:  A GOSIP  V2.0  Compliant  product  may  well  be  IGOSS  Compliant  Indeed,  there 
are  protocols  for  which  IGOSS  may  not  add  any  requirement/functionality  or  any 
testing  requirement 
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